2024-09-12T02:43:11.097 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T02:43:11.126 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:43:11.217 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901648' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ0PxpZY/bbC5kY9YHSWCJz8pHWRE/dvSOf4/pfbogdxvoYu3ih4u5g2Axh2KnV7RYjkVq4imPHeTd3krmktgws= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8gzzzIjU+OxXogWK5NNf6uewh5GJiaFUEZxJGD9T03BIn/fdnNLyCCBNaiFq7ii+JuCFSrKuREAdjo4aNGV6g= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxBWknOy9r/L84IObXR7zpgYHVmAleQQgTcCHAapuzDEOHwyqYTbYVV5TDf4490CPudVSLaAa8gktbdD9njwzM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T02:43:11.218 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T02:43:11.218 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T02:43:11.218 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T02:43:11.220 INFO:teuthology.task.internal:Checking packages... 2024-09-12T02:43:11.241 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T02:43:11.241 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T02:43:11.241 INFO:teuthology.packaging:ref: None 2024-09-12T02:43:11.241 INFO:teuthology.packaging:tag: None 2024-09-12T02:43:11.241 INFO:teuthology.packaging:branch: squid 2024-09-12T02:43:11.241 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:43:11.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T02:43:11.446 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T02:43:11.448 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T02:43:11.455 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T02:43:11.455 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T02:43:11.469 INFO:teuthology.task.internal:Saving configuration 2024-09-12T02:43:11.489 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T02:43:11.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T02:43:11.525 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi057.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:52.586367', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ0PxpZY/bbC5kY9YHSWCJz8pHWRE/dvSOf4/pfbogdxvoYu3ih4u5g2Axh2KnV7RYjkVq4imPHeTd3krmktgws='} 2024-09-12T02:43:11.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:52.584620', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB8gzzzIjU+OxXogWK5NNf6uewh5GJiaFUEZxJGD9T03BIn/fdnNLyCCBNaiFq7ii+JuCFSrKuREAdjo4aNGV6g='} 2024-09-12T02:43:11.567 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:37:52.582582', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxBWknOy9r/L84IObXR7zpgYHVmAleQQgTcCHAapuzDEOHwyqYTbYVV5TDf4490CPudVSLaAa8gktbdD9njwzM='} 2024-09-12T02:43:11.568 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T02:43:11.578 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T02:43:11.578 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T02:43:11.578 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T02:43:11.578 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T02:43:11.674 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19291cfbe0>, signals=[15]) 2024-09-12T02:43:11.674 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T02:43:11.686 INFO:teuthology.task.internal:Opening connections... 2024-09-12T02:43:11.686 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T02:43:11.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:43:11.764 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-09-12T02:43:11.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:43:11.845 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-09-12T02:43:11.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:43:11.940 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T02:43:11.952 DEBUG:teuthology.orchestra.run.smithi057:> uname -m 2024-09-12T02:43:11.971 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2024-09-12T02:43:11.971 DEBUG:teuthology.orchestra.run.smithi057:> cat /etc/os-release 2024-09-12T02:43:12.030 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Stream" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:43:12.031 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:43:12.032 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2024-09-12T02:43:12.049 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-09-12T02:43:12.068 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-09-12T02:43:12.069 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-09-12T02:43:12.132 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:43:12.133 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:43:12.134 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:43:12.135 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:43:12.135 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-09-12T02:43:12.158 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-09-12T02:43:12.177 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-09-12T02:43:12.178 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-09-12T02:43:12.243 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-09-12T02:43:12.243 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:43:12.244 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:43:12.245 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-09-12T02:43:12.277 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T02:43:12.312 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T02:43:12.455 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T02:43:12.455 DEBUG:teuthology.orchestra.run.smithi057:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:43:12.458 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:43:12.461 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:43:12.478 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T02:43:12.840 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T02:43:12.840 DEBUG:teuthology.orchestra.run.smithi057:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:43:12.842 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:43:12.844 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:43:12.952 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T02:43:13.051 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T02:43:13.066 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T02:43:13.066 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-12T02:43:13.067 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-12T02:43:13.067 DEBUG:teuthology.orchestra.run.smithi057:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:43:13.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:43:13.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-12T02:43:13.253 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T02:43:13.253 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-12T02:43:13.253 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-09-12T02:43:13.270 INFO:teuthology.orchestra.run.smithi057.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:43:13.270 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T02:43:13.271 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-12T02:43:13.271 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum upgrade -y linux-firmware 2024-09-12T02:43:16.665 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-12T02:43:25.440 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-09-12T02:43:30.115 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.5 MB 00:00 2024-09-12T02:43:31.641 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-12T02:43:32.612 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-12T02:43:38.326 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:43:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-09-12T02:43:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:Nothing to do. 2024-09-12T02:43:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-09-12T02:43:40.604 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T02:43:40.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T02:43:40.748 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-12T02:43:40.748 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-12T02:43:40.751 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi057.front.sepia.ceph.com to complete... 2024-09-12T02:43:49.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12 02:43:49 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-12T02:43:49.781 INFO:teuthology.orchestra.run.smithi057.stderr:FINISHED --2024-09-12 02:43:49-- 2024-09-12T02:43:49.781 INFO:teuthology.orchestra.run.smithi057.stderr:Total wall clock time: 9.0s 2024-09-12T02:43:49.781 INFO:teuthology.orchestra.run.smithi057.stderr:Downloaded: 1 files, 343M in 8.7s (39.3 MB/s) 2024-09-12T02:43:49.784 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-12T02:43:49.784 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-12T02:43:51.992 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying... ######################################## 2024-09-12T02:43:52.111 INFO:teuthology.orchestra.run.smithi057.stdout:Preparing... ######################################## 2024-09-12T02:43:52.215 INFO:teuthology.orchestra.run.smithi057.stdout:Updating / installing... 2024-09-12T02:44:00.562 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-12T02:44:40.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi057.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-12T02:44:40.349 DEBUG:teuthology.orchestra.run.smithi057:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-12T02:44:40.430 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-12T02:44:40.431 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-12T02:44:40.432 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-12T02:44:40.433 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-12T02:44:40.434 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-12T02:44:40.435 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-12T02:44:40.436 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-12T02:44:40.437 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-12T02:44:40.438 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-12T02:44:40.439 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-12T02:44:40.440 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-12T02:44:40.441 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-12T02:44:40.442 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-12T02:44:40.443 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-12T02:44:40.444 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-12T02:44:40.445 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-12T02:44:40.446 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-12T02:44:40.447 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-12T02:44:40.448 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-12T02:44:40.449 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-12T02:44:40.450 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-12T02:44:40.451 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-12T02:44:40.452 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-12T02:44:40.453 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-12T02:44:40.454 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-12T02:44:40.455 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-12T02:44:40.456 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-12T02:44:40.457 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-12T02:44:40.458 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-12T02:44:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-12T02:44:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-12T02:44:40.461 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-12T02:44:40.462 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-12T02:44:40.463 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-12T02:44:40.464 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-12T02:44:40.465 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-12T02:44:40.466 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-12T02:44:40.467 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-12T02:44:40.468 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-12T02:44:40.469 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-12T02:44:40.470 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-12T02:44:40.471 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-12T02:44:40.472 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-12T02:44:40.473 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-12T02:44:40.474 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-12T02:44:40.475 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-12T02:44:40.476 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-12T02:44:40.477 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-12T02:44:40.478 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-12T02:44:40.479 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-12T02:44:40.480 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-12T02:44:40.481 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-12T02:44:40.482 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-12T02:44:40.483 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-12T02:44:40.484 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-12T02:44:40.485 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-12T02:44:40.486 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-12T02:44:40.487 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-12T02:44:40.488 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-12T02:44:40.489 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-12T02:44:40.490 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-12T02:44:40.491 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-12T02:44:40.492 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-12T02:44:40.493 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-12T02:44:40.494 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-12T02:44:40.495 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-12T02:44:40.496 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-12T02:44:40.497 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-12T02:44:40.498 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-12T02:44:40.499 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-12T02:44:40.500 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-12T02:44:40.501 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-12T02:44:40.502 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-12T02:44:40.503 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-12T02:44:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-12T02:44:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-12T02:44:40.506 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-12T02:44:40.507 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-12T02:44:40.508 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-12T02:44:40.509 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-12T02:44:40.510 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-12T02:44:40.511 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-12T02:44:40.512 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-12T02:44:40.513 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-12T02:44:40.514 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-12T02:44:40.515 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-12T02:44:40.516 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-12T02:44:40.517 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-12T02:44:40.518 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-12T02:44:40.519 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-12T02:44:40.520 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-12T02:44:40.521 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-12T02:44:40.522 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-12T02:44:40.523 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-12T02:44:40.524 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-12T02:44:40.525 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-12T02:44:40.526 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-12T02:44:40.527 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-12T02:44:40.528 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-12T02:44:40.529 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-12T02:44:40.530 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-12T02:44:40.531 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-12T02:44:40.532 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-12T02:44:40.533 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-12T02:44:40.534 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-12T02:44:40.535 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-12T02:44:40.536 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-12T02:44:40.537 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-12T02:44:40.538 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-12T02:44:40.539 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-12T02:44:40.540 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-12T02:44:40.541 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-12T02:44:40.542 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-12T02:44:40.543 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-12T02:44:40.544 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-12T02:44:40.545 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-12T02:44:40.546 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-12T02:44:40.547 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-12T02:44:40.548 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-12T02:44:40.549 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-12T02:44:40.550 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-12T02:44:40.551 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-12T02:44:40.552 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-12T02:44:40.553 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-12T02:44:40.554 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-12T02:44:40.555 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-12T02:44:40.556 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-12T02:44:40.557 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-12T02:44:40.558 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-12T02:44:40.559 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-12T02:44:40.560 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-12T02:44:40.561 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-12T02:44:40.562 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-12T02:44:40.563 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-12T02:44:40.564 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-12T02:44:40.565 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-12T02:44:40.566 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-12T02:44:40.567 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-12T02:44:40.568 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-12T02:44:40.569 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-12T02:44:40.570 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-12T02:44:40.571 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-12T02:44:40.572 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-12T02:44:40.573 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-12T02:44:40.574 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-12T02:44:40.575 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-12T02:44:40.575 INFO:teuthology.orchestra.run.smithi057.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T02:44:40.578 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.579 DEBUG:teuthology.orchestra.run.smithi057:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-12T02:44:40.603 INFO:teuthology.orchestra.run.smithi057.stdout:postinstall scriptlet (using /bin/sh): 2024-09-12T02:44:40.603 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T02:44:40.603 INFO:teuthology.orchestra.run.smithi057.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T02:44:40.603 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:for file in vmlinuz System.map config; do 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: fi 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:done 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.604 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-09-12T02:44:40.605 INFO:teuthology.orchestra.run.smithi057.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-12T02:44:40.605 INFO:teuthology.orchestra.run.smithi057.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T02:44:40.605 INFO:teuthology.orchestra.run.smithi057.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.605 INFO:teuthology.orchestra.run.smithi057.stdout:fi 2024-09-12T02:44:40.606 DEBUG:teuthology.orchestra.run.smithi057:> sudo rpm -qi grub2-tools 2024-09-12T02:44:40.690 INFO:teuthology.orchestra.run.smithi057.stdout:Name : grub2-tools 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Epoch : 1 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Version : 2.06 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Release : 82.el9 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Architecture: x86_64 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Group : Unspecified 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Size : 8274670 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:License : GPLv3+ 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:44:40.691 INFO:teuthology.orchestra.run.smithi057.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:Packager : builder@centos.org 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:Vendor : CentOS 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:Summary : Support tools for GRUB. 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:Description : 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:hardware devices. 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:44:40.692 INFO:teuthology.orchestra.run.smithi057.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:44:40.693 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:44:40.693 INFO:teuthology.task.kernel:Updating grub on smithi057 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:40.693 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:44:41.236 INFO:teuthology.orchestra.run.smithi057.stderr:Generating grub configuration file ... 2024-09-12T02:44:42.753 INFO:teuthology.orchestra.run.smithi057.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:44:42.773 INFO:teuthology.orchestra.run.smithi057.stderr:done 2024-09-12T02:44:42.775 DEBUG:teuthology.orchestra.run.smithi057:> mktemp 2024-09-12T02:44:42.830 INFO:teuthology.orchestra.run.smithi057.stdout:/tmp/tmp.zvj4dWedwF 2024-09-12T02:44:42.830 DEBUG:teuthology.orchestra.run.smithi057:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zvj4dWedwF 2024-09-12T02:44:42.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0666 /tmp/tmp.zvj4dWedwF 2024-09-12T02:44:43.035 DEBUG:teuthology.orchestra.remote:smithi057:/tmp/tmp.zvj4dWedwF is 6KB 2024-09-12T02:44:43.082 DEBUG:teuthology.orchestra.run.smithi057:> rm -fr /tmp/tmp.zvj4dWedwF 2024-09-12T02:44:43.097 DEBUG:teuthology.orchestra.run.smithi057:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:44:43.159 INFO:teuthology.orchestra.run.smithi057.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:44:43.159 INFO:teuthology.orchestra.run.smithi057.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:44:43.159 INFO:teuthology.orchestra.run.smithi057.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:44:43.159 INFO:teuthology.orchestra.run.smithi057.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-12T02:44:43.160 DEBUG:teuthology.orchestra.run.smithi057:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:44:43.356 DEBUG:teuthology.orchestra.run.smithi057:> sudo shutdown -r now 2024-09-12T02:45:13.359 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T02:45:13.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:45:13.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:31.837 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-09-12T02:45:40.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:45:40.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:43.905 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.57 2024-09-12T02:45:55.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:45:55.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:46:55.909 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T02:47:10.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:47:10.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:47:11.207 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-09-12T02:47:11.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:47:11.699 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-12T02:47:11.699 DEBUG:teuthology.orchestra.run.smithi057:> uname -r 2024-09-12T02:47:11.733 INFO:teuthology.orchestra.run.smithi057.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:47:11.733 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T02:47:11.733 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:47:11.734 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:47:12.735 DEBUG:teuthology.parallel:result is None 2024-09-12T02:47:12.735 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T02:47:12.742 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T02:47:12.742 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:47:12.746 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:47:12.749 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:47:12.780 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T02:47:12.788 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T02:47:12.797 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T02:47:12.798 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:47:12.825 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:47:12.828 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:47:12.925 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T02:47:12.932 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T02:47:12.932 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:47:12.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:47:12.950 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:47:12.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:47:12.967 DEBUG:teuthology.orchestra.run.smithi057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:47:12.971 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:47:12.993 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:47:13.027 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.045 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.062 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.073 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.083 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.102 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:47:13.105 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T02:47:13.112 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T02:47:13.112 DEBUG:teuthology.orchestra.run.smithi057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:47:13.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:47:13.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:47:13.170 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T02:47:13.180 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T02:47:13.180 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:47:13.185 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:47:13.188 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:47:13.215 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:13.319 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:13.410 DEBUG:teuthology.orchestra.run.smithi057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:13.474 DEBUG:teuthology.orchestra.run.smithi057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:13.544 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:47:13.544 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:47:13.598 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:13.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:13.697 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:13.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:13.825 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:47:13.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:47:13.892 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:13.937 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:47:14.000 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:14.044 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:47:14.089 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:47:14.089 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:47:14.156 DEBUG:teuthology.orchestra.run.smithi057:> sudo service rsyslog restart 2024-09-12T02:47:14.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-09-12T02:47:14.163 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-09-12T02:47:14.200 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:47:14.234 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:47:14.247 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:47:14.683 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T02:47:14.691 INFO:teuthology.task.internal:Starting timer... 2024-09-12T02:47:14.691 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T02:47:14.702 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T02:47:14.709 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T02:47:14.710 DEBUG:teuthology.orchestra.run.smithi057:> sudo service auditd rotate 2024-09-12T02:47:14.934 INFO:teuthology.orchestra.run.smithi057.stdout:Rotating logs: 2024-09-12T02:47:14.940 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2024-09-12T02:47:15.006 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2024-09-12T02:47:15.008 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-09-12T02:47:15.075 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-09-12T02:47:15.077 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T02:47:15.078 DEBUG:teuthology.orchestra.run.smithi057:> /usr/sbin/getenforce 2024-09-12T02:47:15.161 INFO:teuthology.orchestra.run.smithi057.stdout:Permissive 2024-09-12T02:47:15.161 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2024-09-12T02:47:15.183 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2024-09-12T02:47:15.183 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-09-12T02:47:15.214 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-09-12T02:47:15.214 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi057.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-09-12T02:47:15.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:47:15.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:47:15.273 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:47:15.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:47:15.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:47:15.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:47:15.330 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T02:47:15.330 DEBUG:teuthology.orchestra.run.smithi057:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:47:15.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:47:15.424 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:47:15.457 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T02:47:15.466 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T02:47:15.469 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T02:47:15.605 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T02:47:15.616 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T02:47:15.617 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi057.front.sepia.ceph.com,smithi181.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T02:51:57.619 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi057.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-09-12T02:51:57.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:51:57.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:51:57.712 DEBUG:teuthology.orchestra.run.smithi057:> true 2024-09-12T02:51:57.817 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi057.front.sepia.ceph.com' 2024-09-12T02:51:57.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-12T02:51:57.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:51:57.895 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-12T02:51:57.984 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-12T02:51:57.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-12T02:51:57.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:51:58.061 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-12T02:51:58.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-12T02:51:58.146 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T02:51:58.156 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T02:51:58.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:51:58.156 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:58.161 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:51:58.161 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:58.163 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:51:58.163 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:58.191 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:51:58.205 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:51:58.214 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:51:58.220 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:51:58.229 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:51:58.229 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2024-09-12T02:51:58.240 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2024-09-12T02:51:58.250 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:51:58.253 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-09-12T02:51:58.254 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:51:58.264 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-09-12T02:51:58.268 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:51:58.278 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:51:58.292 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:51:58.301 INFO:teuthology.orchestra.run.smithi057.stderr:sudo: ntpd: command not found 2024-09-12T02:51:58.325 INFO:teuthology.orchestra.run.smithi057.stdout:506 Cannot talk to daemon 2024-09-12T02:51:58.331 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.335 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.354 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:51:58.359 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-12T02:51:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.362 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.363 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.385 INFO:teuthology.orchestra.run.smithi057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:51:58.488 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:58.494 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-09-12T02:51:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.495 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:58.496 INFO:teuthology.run_tasks:Running task install... 2024-09-12T02:51:58.503 DEBUG:teuthology.task.install:project ceph 2024-09-12T02:51:58.504 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T02:51:58.504 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T02:51:58.504 INFO:teuthology.task.install:Using flavor: default 2024-09-12T02:51:58.509 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T02:51:58.509 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T02:51:58.509 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:51:58.510 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:58.511 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:51:58.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:58.512 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:51:58.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:51:58.657 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:51:58.658 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:51:58.661 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:51:58.661 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:51:58.678 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:51:58.679 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:51:58.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:51:58.768 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:51:58.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:51:58.776 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:51:58.776 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:51:58.776 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:51:58.787 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:51:58.787 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:51:58.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:51:58.796 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:51:58.796 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:51:58.815 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:51:58.815 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:51:58.833 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:51:58.833 DEBUG:teuthology.orchestra.run.smithi057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:51:58.862 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:51:58.881 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:51:58.931 DEBUG:teuthology.orchestra.run.smithi057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:51:58.938 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:51:58.958 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:51:59.003 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-09-12T02:51:59.005 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-09-12T02:51:59.010 DEBUG:teuthology.orchestra.run.smithi057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:51:59.022 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2024-09-12T02:51:59.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2024-09-12T02:51:59.064 INFO:teuthology.orchestra.run.smithi057.stdout:check_obsoletes = 1 2024-09-12T02:51:59.067 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum clean all 2024-09-12T02:51:59.230 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-09-12T02:51:59.249 INFO:teuthology.orchestra.run.smithi181.stdout:45 files removed 2024-09-12T02:51:59.258 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:51:59.276 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:51:59.342 INFO:teuthology.orchestra.run.smithi057.stdout:45 files removed 2024-09-12T02:51:59.375 DEBUG:teuthology.orchestra.run.smithi057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:51:59.799 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-12T02:51:59.822 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-12T02:52:00.066 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-12T02:52:00.106 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T02:52:00.139 INFO:teuthology.orchestra.run.smithi057.stdout:ceph packages for x86_64 234 kB/s | 84 kB 00:00 2024-09-12T02:52:00.325 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-12T02:52:00.373 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T02:52:00.456 INFO:teuthology.orchestra.run.smithi057.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T02:52:00.766 INFO:teuthology.orchestra.run.smithi057.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-09-12T02:52:00.848 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-12T02:52:01.009 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-12T02:52:02.945 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-12T02:52:03.457 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-09-12T02:52:03.798 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-12T02:52:06.080 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-12T02:52:07.867 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T02:52:08.478 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-12T02:52:09.759 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-12T02:52:09.928 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-12T02:52:10.791 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-12T02:52:10.872 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-12T02:52:13.349 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.5 MB 00:03 2024-09-12T02:52:15.233 INFO:teuthology.orchestra.run.smithi057.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-12T02:52:16.263 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T02:52:16.386 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:52:16.454 INFO:teuthology.orchestra.run.smithi057.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-12T02:52:18.557 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.557 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.568 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:52:18.569 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.570 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:52:18.571 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.572 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:52:18.574 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.575 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.638 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-12T02:52:18.655 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.656 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:52:18.663 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:52:18.664 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:52:18.665 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:52:18.666 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:52:18.667 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:52:18.668 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:52:18.669 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:52:18.670 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:52:18.671 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:52:18.672 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:52:18.673 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:52:18.674 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:52:18.675 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:52:18.676 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-09-12T02:52:18.677 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:52:18.678 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 226 M 2024-09-12T02:52:18.678 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.679 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:52:18.737 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2024-09-12T02:52:18.751 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-12T02:52:18.751 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:52:18.752 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:52:18.753 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:52:18.754 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2024-09-12T02:52:18.755 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:52:18.756 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:52:18.757 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:52:18.758 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:52:18.759 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:52:18.760 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:52:18.761 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:52:18.762 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:52:18.763 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:52:18.764 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================== 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout:Install 148 Packages 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 8 Packages 2024-09-12T02:52:18.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:52:18.766 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 226 M 2024-09-12T02:52:18.766 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2024-09-12T02:52:19.204 INFO:teuthology.orchestra.run.smithi181.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-12T02:52:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-12T02:52:19.362 INFO:teuthology.orchestra.run.smithi181.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 5.5 MB/s | 897 kB 00:00 2024-09-12T02:52:19.454 INFO:teuthology.orchestra.run.smithi181.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T02:52:19.665 INFO:teuthology.orchestra.run.smithi204.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-12T02:52:19.713 INFO:teuthology.orchestra.run.smithi181.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 8.5 MB/s | 5.4 MB 00:00 2024-09-12T02:52:19.732 INFO:teuthology.orchestra.run.smithi204.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-12T02:52:19.795 INFO:teuthology.orchestra.run.smithi181.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 2.4 MB 00:00 2024-09-12T02:52:19.864 INFO:teuthology.orchestra.run.smithi181.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 1.5 MB 00:00 2024-09-12T02:52:19.891 INFO:teuthology.orchestra.run.smithi204.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.0 MB/s | 5.4 MB 00:00 2024-09-12T02:52:20.264 INFO:teuthology.orchestra.run.smithi181.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T02:52:20.391 INFO:teuthology.orchestra.run.smithi204.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-09-12T02:52:20.600 INFO:teuthology.orchestra.run.smithi204.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 2.7 MB/s | 2.4 MB 00:00 2024-09-12T02:52:20.639 INFO:teuthology.orchestra.run.smithi181.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-12T02:52:20.706 INFO:teuthology.orchestra.run.smithi181.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-12T02:52:20.867 INFO:teuthology.orchestra.run.smithi204.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 9.9 MB/s | 4.7 MB 00:00 2024-09-12T02:52:21.057 INFO:teuthology.orchestra.run.smithi181.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-12T02:52:21.182 INFO:teuthology.orchestra.run.smithi181.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 12 MB/s | 11 MB 00:00 2024-09-12T02:52:21.207 INFO:teuthology.orchestra.run.smithi181.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.5 MB/s | 228 kB 00:00 2024-09-12T02:52:21.249 INFO:teuthology.orchestra.run.smithi181.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 405 kB/s | 27 kB 00:00 2024-09-12T02:52:21.274 INFO:teuthology.orchestra.run.smithi181.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 473 kB/s | 32 kB 00:00 2024-09-12T02:52:21.333 INFO:teuthology.orchestra.run.smithi181.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-12T02:52:21.358 INFO:teuthology.orchestra.run.smithi181.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T02:52:21.400 INFO:teuthology.orchestra.run.smithi181.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.8 MB/s | 127 kB 00:00 2024-09-12T02:52:21.442 INFO:teuthology.orchestra.run.smithi181.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-12T02:52:21.500 INFO:teuthology.orchestra.run.smithi181.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 773 kB/s | 45 kB 00:00 2024-09-12T02:52:21.568 INFO:teuthology.orchestra.run.smithi181.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T02:52:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 8.4 MB/s | 21 MB 00:02 2024-09-12T02:52:21.644 INFO:teuthology.orchestra.run.smithi181.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-12T02:52:21.693 INFO:teuthology.orchestra.run.smithi204.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-12T02:52:21.728 INFO:teuthology.orchestra.run.smithi181.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-12T02:52:21.811 INFO:teuthology.orchestra.run.smithi181.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-12T02:52:21.835 INFO:teuthology.orchestra.run.smithi204.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 11 MB/s | 11 MB 00:00 2024-09-12T02:52:21.837 INFO:teuthology.orchestra.run.smithi181.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.7 MB/s | 301 kB 00:00 2024-09-12T02:52:21.862 INFO:teuthology.orchestra.run.smithi181.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T02:52:21.893 INFO:teuthology.orchestra.run.smithi204.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T02:52:21.896 INFO:teuthology.orchestra.run.smithi181.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-12T02:52:21.943 INFO:teuthology.orchestra.run.smithi204.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 543 kB/s | 27 kB 00:00 2024-09-12T02:52:21.971 INFO:teuthology.orchestra.run.smithi181.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-12T02:52:22.010 INFO:teuthology.orchestra.run.smithi204.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 475 kB/s | 32 kB 00:00 2024-09-12T02:52:22.046 INFO:teuthology.orchestra.run.smithi181.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 425 kB/s | 32 kB 00:00 2024-09-12T02:52:22.121 INFO:teuthology.orchestra.run.smithi181.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-12T02:52:22.136 INFO:teuthology.orchestra.run.smithi204.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 752 kB 00:00 2024-09-12T02:52:22.147 INFO:teuthology.orchestra.run.smithi181.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.4 MB/s | 146 kB 00:00 2024-09-12T02:52:22.194 INFO:teuthology.orchestra.run.smithi204.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T02:52:22.295 INFO:teuthology.orchestra.run.smithi204.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.2 MB/s | 127 kB 00:00 2024-09-12T02:52:22.395 INFO:teuthology.orchestra.run.smithi204.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.0 MB/s | 506 kB 00:00 2024-09-12T02:52:22.464 INFO:teuthology.orchestra.run.smithi181.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 10 MB/s | 3.6 MB 00:00 2024-09-12T02:52:22.564 INFO:teuthology.orchestra.run.smithi181.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-12T02:52:22.714 INFO:teuthology.orchestra.run.smithi181.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-12T02:52:22.739 INFO:teuthology.orchestra.run.smithi181.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 282 kB/s | 49 kB 00:00 2024-09-12T02:52:22.781 INFO:teuthology.orchestra.run.smithi181.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 247 kB/s | 16 kB 00:00 2024-09-12T02:52:22.815 INFO:teuthology.orchestra.run.smithi181.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.7 MB/s | 285 kB 00:00 2024-09-12T02:52:22.874 INFO:teuthology.orchestra.run.smithi181.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-12T02:52:22.899 INFO:teuthology.orchestra.run.smithi181.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 218 kB/s | 18 kB 00:00 2024-09-12T02:52:22.937 INFO:teuthology.orchestra.run.smithi204.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 9.9 MB/s | 5.3 MB 00:00 2024-09-12T02:52:22.995 INFO:teuthology.orchestra.run.smithi204.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 775 kB/s | 45 kB 00:00 2024-09-12T02:52:23.046 INFO:teuthology.orchestra.run.smithi204.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-12T02:52:23.051 INFO:teuthology.orchestra.run.smithi057.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-12T02:52:23.096 INFO:teuthology.orchestra.run.smithi204.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T02:52:23.154 INFO:teuthology.orchestra.run.smithi204.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T02:52:23.222 INFO:teuthology.orchestra.run.smithi204.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T02:52:23.280 INFO:teuthology.orchestra.run.smithi204.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T02:52:23.330 INFO:teuthology.orchestra.run.smithi204.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:52:23.449 INFO:teuthology.orchestra.run.smithi181.stdout:(39/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T02:52:23.475 INFO:teuthology.orchestra.run.smithi181.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 133 kB/s | 80 kB 00:00 2024-09-12T02:52:23.531 INFO:teuthology.orchestra.run.smithi204.stdout:(26/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-12T02:52:23.581 INFO:teuthology.orchestra.run.smithi204.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T02:52:23.632 INFO:teuthology.orchestra.run.smithi204.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 639 kB/s | 32 kB 00:00 2024-09-12T02:52:23.650 INFO:teuthology.orchestra.run.smithi181.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-12T02:52:23.675 INFO:teuthology.orchestra.run.smithi181.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 849 kB/s | 191 kB 00:00 2024-09-12T02:52:23.681 INFO:teuthology.orchestra.run.smithi204.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T02:52:23.701 INFO:teuthology.orchestra.run.smithi181.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-12T02:52:23.759 INFO:teuthology.orchestra.run.smithi181.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-12T02:52:23.784 INFO:teuthology.orchestra.run.smithi181.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T02:52:23.843 INFO:teuthology.orchestra.run.smithi181.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T02:52:23.877 INFO:teuthology.orchestra.run.smithi181.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-12T02:52:23.890 INFO:teuthology.orchestra.run.smithi204.stdout:(30/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 17 MB/s | 3.6 MB 00:00 2024-09-12T02:52:24.002 INFO:teuthology.orchestra.run.smithi181.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-09-12T02:52:24.044 INFO:teuthology.orchestra.run.smithi181.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-12T02:52:24.078 INFO:teuthology.orchestra.run.smithi181.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-12T02:52:24.094 INFO:teuthology.orchestra.run.smithi181.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-12T02:52:24.137 INFO:teuthology.orchestra.run.smithi181.stdout:(52/156): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-09-12T02:52:24.154 INFO:teuthology.orchestra.run.smithi181.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-12T02:52:24.174 INFO:teuthology.orchestra.run.smithi204.stdout:(31/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 4.7 MB/s | 17 MB 00:03 2024-09-12T02:52:24.187 INFO:teuthology.orchestra.run.smithi181.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T02:52:24.204 INFO:teuthology.orchestra.run.smithi181.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T02:52:24.221 INFO:teuthology.orchestra.run.smithi181.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-09-12T02:52:24.258 INFO:teuthology.orchestra.run.smithi204.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.9 MB/s | 249 kB 00:00 2024-09-12T02:52:24.308 INFO:teuthology.orchestra.run.smithi181.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-09-12T02:52:24.316 INFO:teuthology.orchestra.run.smithi204.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 847 kB/s | 49 kB 00:00 2024-09-12T02:52:24.363 INFO:teuthology.orchestra.run.smithi181.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-09-12T02:52:24.404 INFO:teuthology.orchestra.run.smithi204.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-12T02:52:24.405 INFO:teuthology.orchestra.run.smithi181.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.7 MB/s | 662 kB 00:00 2024-09-12T02:52:24.428 INFO:teuthology.orchestra.run.smithi204.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 148 kB/s | 16 kB 00:00 2024-09-12T02:52:24.447 INFO:teuthology.orchestra.run.smithi181.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T02:52:24.453 INFO:teuthology.orchestra.run.smithi204.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-12T02:52:24.472 INFO:teuthology.orchestra.run.smithi181.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-12T02:52:24.498 INFO:teuthology.orchestra.run.smithi181.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-12T02:52:24.504 INFO:teuthology.orchestra.run.smithi204.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 364 kB/s | 18 kB 00:00 2024-09-12T02:52:24.523 INFO:teuthology.orchestra.run.smithi181.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T02:52:24.548 INFO:teuthology.orchestra.run.smithi181.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T02:52:24.562 INFO:teuthology.orchestra.run.smithi204.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.6 MB/s | 765 kB 00:00 2024-09-12T02:52:24.573 INFO:teuthology.orchestra.run.smithi181.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-12T02:52:24.632 INFO:teuthology.orchestra.run.smithi181.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-12T02:52:24.688 INFO:teuthology.orchestra.run.smithi204.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 435 kB/s | 80 kB 00:00 2024-09-12T02:52:24.699 INFO:teuthology.orchestra.run.smithi181.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-12T02:52:24.783 INFO:teuthology.orchestra.run.smithi181.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-12T02:52:24.825 INFO:teuthology.orchestra.run.smithi181.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-12T02:52:24.850 INFO:teuthology.orchestra.run.smithi181.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T02:52:24.917 INFO:teuthology.orchestra.run.smithi181.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-12T02:52:24.959 INFO:teuthology.orchestra.run.smithi181.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-12T02:52:25.034 INFO:teuthology.orchestra.run.smithi181.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-09-12T02:52:25.109 INFO:teuthology.orchestra.run.smithi181.stdout:(74/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.8 MB/s | 4.9 MB 00:00 2024-09-12T02:52:25.134 INFO:teuthology.orchestra.run.smithi181.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 612 kB/s | 61 kB 00:00 2024-09-12T02:52:25.155 INFO:teuthology.orchestra.run.smithi204.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 14 MB/s | 49 MB 00:03 2024-09-12T02:52:25.160 INFO:teuthology.orchestra.run.smithi181.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-12T02:52:25.180 INFO:teuthology.orchestra.run.smithi204.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 388 kB/s | 191 kB 00:00 2024-09-12T02:52:25.185 INFO:teuthology.orchestra.run.smithi181.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T02:52:25.202 INFO:teuthology.orchestra.run.smithi181.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-09-12T02:52:25.222 INFO:teuthology.orchestra.run.smithi204.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-12T02:52:25.228 INFO:teuthology.orchestra.run.smithi181.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 455 kB/s | 19 kB 00:00 2024-09-12T02:52:25.261 INFO:teuthology.orchestra.run.smithi181.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T02:52:25.272 INFO:teuthology.orchestra.run.smithi204.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T02:52:25.278 INFO:teuthology.orchestra.run.smithi181.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-12T02:52:25.320 INFO:teuthology.orchestra.run.smithi181.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-12T02:52:25.331 INFO:teuthology.orchestra.run.smithi204.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T02:52:25.356 INFO:teuthology.orchestra.run.smithi204.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-12T02:52:25.382 INFO:teuthology.orchestra.run.smithi204.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T02:52:25.395 INFO:teuthology.orchestra.run.smithi181.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-12T02:52:25.420 INFO:teuthology.orchestra.run.smithi181.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-12T02:52:25.440 INFO:teuthology.orchestra.run.smithi204.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T02:52:25.446 INFO:teuthology.orchestra.run.smithi181.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T02:52:25.463 INFO:teuthology.orchestra.run.smithi181.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T02:52:25.473 INFO:teuthology.orchestra.run.smithi204.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-09-12T02:52:25.488 INFO:teuthology.orchestra.run.smithi181.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-12T02:52:25.513 INFO:teuthology.orchestra.run.smithi181.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T02:52:25.539 INFO:teuthology.orchestra.run.smithi181.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T02:52:25.556 INFO:teuthology.orchestra.run.smithi181.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-12T02:52:25.572 INFO:teuthology.orchestra.run.smithi181.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-12T02:52:25.623 INFO:teuthology.orchestra.run.smithi181.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-12T02:52:25.641 INFO:teuthology.orchestra.run.smithi057.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:25.642 INFO:teuthology.orchestra.run.smithi057.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:52:25.655 INFO:teuthology.orchestra.run.smithi057.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:52:25.656 INFO:teuthology.orchestra.run.smithi057.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:52:25.656 INFO:teuthology.orchestra.run.smithi181.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-12T02:52:25.657 INFO:teuthology.orchestra.run.smithi057.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:52:25.658 INFO:teuthology.orchestra.run.smithi057.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:52:25.659 INFO:teuthology.orchestra.run.smithi057.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:52:25.662 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:52:25.663 INFO:teuthology.orchestra.run.smithi057.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:52:25.687 INFO:teuthology.orchestra.run.smithi204.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 501 kB/s | 107 kB 00:00 2024-09-12T02:52:25.732 INFO:teuthology.orchestra.run.smithi181.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T02:52:25.737 INFO:teuthology.orchestra.run.smithi057.stdout:Dependencies resolved. 2024-09-12T02:52:25.741 INFO:teuthology.orchestra.run.smithi204.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 407 kB/s | 22 kB 00:00 2024-09-12T02:52:25.757 INFO:teuthology.orchestra.run.smithi181.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-12T02:52:25.761 INFO:teuthology.orchestra.run.smithi057.stdout:====================================================================================== 2024-09-12T02:52:25.761 INFO:teuthology.orchestra.run.smithi057.stdout: Package Arch Version Repository Size 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout:====================================================================================== 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout:Installing: 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:52:25.762 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:52:25.763 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:52:25.764 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrading: 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:52:25.765 INFO:teuthology.orchestra.run.smithi057.stdout:Installing dependencies: 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:52:25.766 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:52:25.767 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:52:25.768 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:52:25.769 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:52:25.770 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:52:25.771 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:52:25.772 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:52:25.773 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:52:25.774 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout:Installing weak dependencies: 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction Summary 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout:====================================================================================== 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout:Install 148 Packages 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout:Upgrade 8 Packages 2024-09-12T02:52:25.775 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:52:25.776 INFO:teuthology.orchestra.run.smithi181.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 856 kB/s | 36 kB 00:00 2024-09-12T02:52:25.776 INFO:teuthology.orchestra.run.smithi057.stdout:Total download size: 226 M 2024-09-12T02:52:25.776 INFO:teuthology.orchestra.run.smithi057.stdout:Downloading Packages: 2024-09-12T02:52:25.791 INFO:teuthology.orchestra.run.smithi181.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-12T02:52:25.808 INFO:teuthology.orchestra.run.smithi204.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-09-12T02:52:25.816 INFO:teuthology.orchestra.run.smithi181.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-12T02:52:25.841 INFO:teuthology.orchestra.run.smithi181.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T02:52:25.866 INFO:teuthology.orchestra.run.smithi204.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 563 kB/s | 33 kB 00:00 2024-09-12T02:52:25.875 INFO:teuthology.orchestra.run.smithi181.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T02:52:25.892 INFO:teuthology.orchestra.run.smithi204.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-09-12T02:52:25.900 INFO:teuthology.orchestra.run.smithi181.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T02:52:25.926 INFO:teuthology.orchestra.run.smithi181.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-12T02:52:25.942 INFO:teuthology.orchestra.run.smithi204.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T02:52:25.976 INFO:teuthology.orchestra.run.smithi204.stdout:(55/156): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 943 kB 00:00 2024-09-12T02:52:25.993 INFO:teuthology.orchestra.run.smithi204.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T02:52:26.043 INFO:teuthology.orchestra.run.smithi204.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-09-12T02:52:26.077 INFO:teuthology.orchestra.run.smithi204.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-12T02:52:26.143 INFO:teuthology.orchestra.run.smithi181.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-12T02:52:26.168 INFO:teuthology.orchestra.run.smithi181.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-12T02:52:26.218 INFO:teuthology.orchestra.run.smithi181.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-12T02:52:26.227 INFO:teuthology.orchestra.run.smithi204.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-12T02:52:26.244 INFO:teuthology.orchestra.run.smithi181.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-12T02:52:26.269 INFO:teuthology.orchestra.run.smithi181.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-12T02:52:26.294 INFO:teuthology.orchestra.run.smithi181.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:52:26.294 INFO:teuthology.orchestra.run.smithi204.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-12T02:52:26.311 INFO:teuthology.orchestra.run.smithi204.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-12T02:52:26.320 INFO:teuthology.orchestra.run.smithi181.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:52:26.345 INFO:teuthology.orchestra.run.smithi181.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T02:52:26.363 INFO:teuthology.orchestra.run.smithi204.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 696 kB/s | 47 kB 00:00 2024-09-12T02:52:26.387 INFO:teuthology.orchestra.run.smithi181.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-12T02:52:26.403 INFO:teuthology.orchestra.run.smithi057.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-12T02:52:26.420 INFO:teuthology.orchestra.run.smithi204.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-12T02:52:26.421 INFO:teuthology.orchestra.run.smithi181.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-12T02:52:26.445 INFO:teuthology.orchestra.run.smithi204.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-12T02:52:26.470 INFO:teuthology.orchestra.run.smithi181.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-12T02:52:26.471 INFO:teuthology.orchestra.run.smithi204.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-12T02:52:26.496 INFO:teuthology.orchestra.run.smithi181.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:52:26.521 INFO:teuthology.orchestra.run.smithi181.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:52:26.547 INFO:teuthology.orchestra.run.smithi181.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-12T02:52:26.570 INFO:teuthology.orchestra.run.smithi057.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 5.3 MB/s | 897 kB 00:00 2024-09-12T02:52:26.572 INFO:teuthology.orchestra.run.smithi181.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-12T02:52:26.588 INFO:teuthology.orchestra.run.smithi204.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-09-12T02:52:26.614 INFO:teuthology.orchestra.run.smithi181.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-12T02:52:26.645 INFO:teuthology.orchestra.run.smithi057.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-12T02:52:26.647 INFO:teuthology.orchestra.run.smithi181.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-12T02:52:26.673 INFO:teuthology.orchestra.run.smithi181.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T02:52:26.740 INFO:teuthology.orchestra.run.smithi181.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-12T02:52:26.765 INFO:teuthology.orchestra.run.smithi181.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-12T02:52:26.782 INFO:teuthology.orchestra.run.smithi181.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 652 kB/s | 11 kB 00:00 2024-09-12T02:52:26.804 INFO:teuthology.orchestra.run.smithi057.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-12T02:52:26.804 INFO:teuthology.orchestra.run.smithi204.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-12T02:52:26.807 INFO:teuthology.orchestra.run.smithi181.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-12T02:52:26.824 INFO:teuthology.orchestra.run.smithi181.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-09-12T02:52:26.841 INFO:teuthology.orchestra.run.smithi181.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 691 kB/s | 23 kB 00:00 2024-09-12T02:52:26.858 INFO:teuthology.orchestra.run.smithi181.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-12T02:52:26.879 INFO:teuthology.orchestra.run.smithi057.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-12T02:52:26.883 INFO:teuthology.orchestra.run.smithi181.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 471 kB/s | 20 kB 00:00 2024-09-12T02:52:26.908 INFO:teuthology.orchestra.run.smithi181.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:52:26.934 INFO:teuthology.orchestra.run.smithi181.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:52:26.937 INFO:teuthology.orchestra.run.smithi057.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-12T02:52:26.938 INFO:teuthology.orchestra.run.smithi204.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-12T02:52:26.959 INFO:teuthology.orchestra.run.smithi181.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:52:26.997 INFO:teuthology.orchestra.run.smithi204.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T02:52:27.001 INFO:teuthology.orchestra.run.smithi181.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-12T02:52:27.026 INFO:teuthology.orchestra.run.smithi181.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-12T02:52:27.052 INFO:teuthology.orchestra.run.smithi181.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-12T02:52:27.077 INFO:teuthology.orchestra.run.smithi181.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T02:52:27.102 INFO:teuthology.orchestra.run.smithi181.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-12T02:52:27.128 INFO:teuthology.orchestra.run.smithi181.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-12T02:52:27.147 INFO:teuthology.orchestra.run.smithi204.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-12T02:52:27.153 INFO:teuthology.orchestra.run.smithi181.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:52:27.178 INFO:teuthology.orchestra.run.smithi181.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:52:27.203 INFO:teuthology.orchestra.run.smithi181.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-12T02:52:27.213 INFO:teuthology.orchestra.run.smithi057.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-12T02:52:27.229 INFO:teuthology.orchestra.run.smithi181.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-12T02:52:27.279 INFO:teuthology.orchestra.run.smithi181.stdout:(142/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-12T02:52:27.297 INFO:teuthology.orchestra.run.smithi204.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.8 MB/s | 4.9 MB 00:00 2024-09-12T02:52:27.304 INFO:teuthology.orchestra.run.smithi181.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-12T02:52:27.331 INFO:teuthology.orchestra.run.smithi204.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-12T02:52:27.338 INFO:teuthology.orchestra.run.smithi181.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-12T02:52:27.356 INFO:teuthology.orchestra.run.smithi204.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T02:52:27.363 INFO:teuthology.orchestra.run.smithi181.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-12T02:52:27.388 INFO:teuthology.orchestra.run.smithi181.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:52:27.414 INFO:teuthology.orchestra.run.smithi181.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-12T02:52:27.415 INFO:teuthology.orchestra.run.smithi204.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T02:52:27.473 INFO:teuthology.orchestra.run.smithi204.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T02:52:27.481 INFO:teuthology.orchestra.run.smithi181.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T02:52:27.582 INFO:teuthology.orchestra.run.smithi204.stdout:(76/156): postgresql-server-13.16-1.el9.x86_64. 9.9 MB/s | 5.8 MB 00:00 2024-09-12T02:52:27.607 INFO:teuthology.orchestra.run.smithi204.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-12T02:52:27.641 INFO:teuthology.orchestra.run.smithi204.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-12T02:52:27.666 INFO:teuthology.orchestra.run.smithi204.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-12T02:52:27.700 INFO:teuthology.orchestra.run.smithi204.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T02:52:27.725 INFO:teuthology.orchestra.run.smithi204.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-12T02:52:27.755 INFO:teuthology.orchestra.run.smithi057.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-12T02:52:27.764 INFO:teuthology.orchestra.run.smithi181.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-12T02:52:27.805 INFO:teuthology.orchestra.run.smithi057.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-12T02:52:27.834 INFO:teuthology.orchestra.run.smithi204.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-12T02:52:27.868 INFO:teuthology.orchestra.run.smithi204.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-12T02:52:27.893 INFO:teuthology.orchestra.run.smithi204.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T02:52:27.926 INFO:teuthology.orchestra.run.smithi204.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T02:52:27.952 INFO:teuthology.orchestra.run.smithi204.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-09-12T02:52:27.980 INFO:teuthology.orchestra.run.smithi057.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-12T02:52:27.985 INFO:teuthology.orchestra.run.smithi204.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T02:52:28.011 INFO:teuthology.orchestra.run.smithi204.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T02:52:28.039 INFO:teuthology.orchestra.run.smithi057.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T02:52:28.065 INFO:teuthology.orchestra.run.smithi181.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 4.9 MB/s | 3.2 MB 00:00 2024-09-12T02:52:28.086 INFO:teuthology.orchestra.run.smithi204.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-12T02:52:28.111 INFO:teuthology.orchestra.run.smithi204.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-12T02:52:28.132 INFO:teuthology.orchestra.run.smithi181.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 3.6 MB/s | 2.3 MB 00:00 2024-09-12T02:52:28.136 INFO:teuthology.orchestra.run.smithi204.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-12T02:52:28.162 INFO:teuthology.orchestra.run.smithi204.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-12T02:52:28.382 INFO:teuthology.orchestra.run.smithi181.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.7 MB/s | 249 kB 00:00 2024-09-12T02:52:28.382 INFO:teuthology.orchestra.run.smithi181.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 232 kB/s | 41 kB 00:00 2024-09-12T02:52:28.382 INFO:teuthology.orchestra.run.smithi181.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(12/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 11 MB/s | 11 MB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 170 kB/s | 27 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 639 kB/s | 32 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.5 MB/s | 752 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 163 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.2 MB/s | 127 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi057.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi204.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-12T02:52:28.383 INFO:teuthology.orchestra.run.smithi204.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-12T02:52:28.384 INFO:teuthology.orchestra.run.smithi204.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:52:28.384 INFO:teuthology.orchestra.run.smithi204.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-12T02:52:28.384 INFO:teuthology.orchestra.run.smithi204.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-12T02:52:28.384 INFO:teuthology.orchestra.run.smithi204.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T02:52:28.384 INFO:teuthology.orchestra.run.smithi204.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-12T02:52:28.389 INFO:teuthology.orchestra.run.smithi204.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T02:52:28.415 INFO:teuthology.orchestra.run.smithi204.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T02:52:28.432 INFO:teuthology.orchestra.run.smithi057.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 774 kB/s | 45 kB 00:00 2024-09-12T02:52:28.448 INFO:teuthology.orchestra.run.smithi204.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-12T02:52:28.483 INFO:teuthology.orchestra.run.smithi181.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 1.4 MB 00:00 2024-09-12T02:52:28.516 INFO:teuthology.orchestra.run.smithi057.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T02:52:28.532 INFO:teuthology.orchestra.run.smithi204.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-12T02:52:28.582 INFO:teuthology.orchestra.run.smithi204.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:52:28.600 INFO:teuthology.orchestra.run.smithi057.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-12T02:52:28.616 INFO:teuthology.orchestra.run.smithi204.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T02:52:28.625 INFO:teuthology.orchestra.run.smithi057.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 162 kB 00:00 2024-09-12T02:52:28.641 INFO:teuthology.orchestra.run.smithi204.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-12T02:52:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T02:52:28.693 INFO:teuthology.orchestra.run.smithi057.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T02:52:28.717 INFO:teuthology.orchestra.run.smithi057.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T02:52:28.742 INFO:teuthology.orchestra.run.smithi057.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:52:28.758 INFO:teuthology.orchestra.run.smithi204.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-12T02:52:28.775 INFO:teuthology.orchestra.run.smithi204.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-12T02:52:28.792 INFO:teuthology.orchestra.run.smithi204.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-12T02:52:28.801 INFO:teuthology.orchestra.run.smithi057.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T02:52:28.825 INFO:teuthology.orchestra.run.smithi204.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-12T02:52:28.860 INFO:teuthology.orchestra.run.smithi057.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-12T02:52:28.868 INFO:teuthology.orchestra.run.smithi204.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 893 kB/s | 67 kB 00:00 2024-09-12T02:52:28.885 INFO:teuthology.orchestra.run.smithi057.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 381 kB/s | 32 kB 00:00 2024-09-12T02:52:28.899 INFO:teuthology.orchestra.run.smithi181.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.6 MB/s | 2.4 MB 00:00 2024-09-12T02:52:28.901 INFO:teuthology.orchestra.run.smithi204.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-12T02:52:28.907 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:52:28.907 INFO:teuthology.orchestra.run.smithi181.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-12T02:52:28.936 INFO:teuthology.orchestra.run.smithi057.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T02:52:28.953 INFO:teuthology.orchestra.run.smithi204.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-12T02:52:29.111 INFO:teuthology.orchestra.run.smithi204.stdout:(114/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-09-12T02:52:29.119 INFO:teuthology.orchestra.run.smithi057.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-12T02:52:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 237 kB/s | 43 kB 00:00 2024-09-12T02:52:29.169 INFO:teuthology.orchestra.run.smithi204.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 977 kB/s | 32 kB 00:00 2024-09-12T02:52:29.195 INFO:teuthology.orchestra.run.smithi204.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 242 kB/s | 14 kB 00:00 2024-09-12T02:52:29.195 INFO:teuthology.orchestra.run.smithi057.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.3 MB/s | 249 kB 00:00 2024-09-12T02:52:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.0 MB/s | 172 kB 00:00 2024-09-12T02:52:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-09-12T02:52:29.287 INFO:teuthology.orchestra.run.smithi204.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T02:52:29.303 INFO:teuthology.orchestra.run.smithi057.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-12T02:52:29.312 INFO:teuthology.orchestra.run.smithi204.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T02:52:29.328 INFO:teuthology.orchestra.run.smithi057.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 369 kB/s | 49 kB 00:00 2024-09-12T02:52:29.337 INFO:teuthology.orchestra.run.smithi204.stdout:(122/156): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-09-12T02:52:29.354 INFO:teuthology.orchestra.run.smithi057.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 328 kB/s | 16 kB 00:00 2024-09-12T02:52:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T02:52:29.387 INFO:teuthology.orchestra.run.smithi057.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T02:52:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-12T02:52:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-12T02:52:29.429 INFO:teuthology.orchestra.run.smithi057.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 10 MB/s | 765 kB 00:00 2024-09-12T02:52:29.439 INFO:teuthology.orchestra.run.smithi204.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-12T02:52:29.454 INFO:teuthology.orchestra.run.smithi057.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 273 kB/s | 18 kB 00:00 2024-09-12T02:52:29.464 INFO:teuthology.orchestra.run.smithi204.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-12T02:52:29.489 INFO:teuthology.orchestra.run.smithi204.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-12T02:52:29.515 INFO:teuthology.orchestra.run.smithi204.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:52:29.540 INFO:teuthology.orchestra.run.smithi204.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:52:29.565 INFO:teuthology.orchestra.run.smithi204.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:52:29.571 INFO:teuthology.orchestra.run.smithi057.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-12T02:52:29.607 INFO:teuthology.orchestra.run.smithi204.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-12T02:52:29.632 INFO:teuthology.orchestra.run.smithi204.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T02:52:29.655 INFO:teuthology.orchestra.run.smithi057.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T02:52:29.657 INFO:teuthology.orchestra.run.smithi204.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-12T02:52:29.683 INFO:teuthology.orchestra.run.smithi204.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T02:52:29.697 INFO:teuthology.orchestra.run.smithi057.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-12T02:52:29.717 INFO:teuthology.orchestra.run.smithi204.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 4.0 MB/s | 1.0 MB 00:00 2024-09-12T02:52:29.739 INFO:teuthology.orchestra.run.smithi057.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-12T02:52:29.742 INFO:teuthology.orchestra.run.smithi204.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-12T02:52:29.767 INFO:teuthology.orchestra.run.smithi204.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:52:29.793 INFO:teuthology.orchestra.run.smithi204.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:52:29.818 INFO:teuthology.orchestra.run.smithi204.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 708 kB/s | 36 kB 00:00 2024-09-12T02:52:29.843 INFO:teuthology.orchestra.run.smithi204.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T02:52:29.869 INFO:teuthology.orchestra.run.smithi204.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:52:29.873 INFO:teuthology.orchestra.run.smithi057.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 799 kB/s | 106 kB 00:00 2024-09-12T02:52:29.894 INFO:teuthology.orchestra.run.smithi204.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T02:52:29.911 INFO:teuthology.orchestra.run.smithi204.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-12T02:52:29.943 INFO:teuthology.orchestra.run.smithi057.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-09-12T02:52:29.961 INFO:teuthology.orchestra.run.smithi204.stdout:(145/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-12T02:52:29.973 INFO:teuthology.orchestra.run.smithi057.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-12T02:52:29.995 INFO:teuthology.orchestra.run.smithi204.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T02:52:29.998 INFO:teuthology.orchestra.run.smithi057.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 125 kB 00:00 2024-09-12T02:52:30.021 INFO:teuthology.orchestra.run.smithi204.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-12T02:52:30.079 INFO:teuthology.orchestra.run.smithi204.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-12T02:52:30.158 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2024-09-12T02:52:30.271 INFO:teuthology.orchestra.run.smithi204.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T02:52:30.309 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2024-09-12T02:52:30.309 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2024-09-12T02:52:30.466 INFO:teuthology.orchestra.run.smithi057.stdout:(47/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T02:52:30.499 INFO:teuthology.orchestra.run.smithi057.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 409 kB/s | 215 kB 00:00 2024-09-12T02:52:30.579 INFO:teuthology.orchestra.run.smithi204.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 5.6 MB/s | 3.2 MB 00:00 2024-09-12T02:52:30.683 INFO:teuthology.orchestra.run.smithi057.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 584 kB/s | 107 kB 00:00 2024-09-12T02:52:30.688 INFO:teuthology.orchestra.run.smithi204.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-12T02:52:30.716 INFO:teuthology.orchestra.run.smithi057.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 960 kB/s | 689 kB 00:00 2024-09-12T02:52:30.733 INFO:teuthology.orchestra.run.smithi057.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-12T02:52:30.775 INFO:teuthology.orchestra.run.smithi057.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-12T02:52:30.797 INFO:teuthology.orchestra.run.smithi204.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 3.4 MB/s | 1.8 MB 00:00 2024-09-12T02:52:30.800 INFO:teuthology.orchestra.run.smithi057.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-12T02:52:30.834 INFO:teuthology.orchestra.run.smithi057.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T02:52:30.847 INFO:teuthology.orchestra.run.smithi204.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.1 MB/s | 2.3 MB 00:00 2024-09-12T02:52:30.859 INFO:teuthology.orchestra.run.smithi057.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-12T02:52:30.872 INFO:teuthology.orchestra.run.smithi204.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.3 MB/s | 249 kB 00:00 2024-09-12T02:52:30.893 INFO:teuthology.orchestra.run.smithi057.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 192 kB/s | 11 kB 00:00 2024-09-12T02:52:30.943 INFO:teuthology.orchestra.run.smithi057.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-12T02:52:30.968 INFO:teuthology.orchestra.run.smithi057.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-12T02:52:31.060 INFO:teuthology.orchestra.run.smithi057.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-12T02:52:31.097 INFO:teuthology.orchestra.run.smithi204.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 1.4 MB 00:00 2024-09-12T02:52:31.111 INFO:teuthology.orchestra.run.smithi057.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-12T02:52:31.136 INFO:teuthology.orchestra.run.smithi057.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-12T02:52:31.195 INFO:teuthology.orchestra.run.smithi057.stdout:(62/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.1 MB/s | 3.0 MB 00:00 2024-09-12T02:52:31.220 INFO:teuthology.orchestra.run.smithi057.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-09-12T02:52:31.245 INFO:teuthology.orchestra.run.smithi057.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-12T02:52:31.271 INFO:teuthology.orchestra.run.smithi057.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-12T02:52:31.304 INFO:teuthology.orchestra.run.smithi057.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-12T02:52:31.529 INFO:teuthology.orchestra.run.smithi057.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-12T02:52:31.598 INFO:teuthology.orchestra.run.smithi204.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-09-12T02:52:31.604 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:52:31.604 INFO:teuthology.orchestra.run.smithi204.stdout:Total 17 MB/s | 226 MB 00:12 2024-09-12T02:52:31.663 INFO:teuthology.orchestra.run.smithi057.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-12T02:52:31.705 INFO:teuthology.orchestra.run.smithi057.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-12T02:52:31.755 INFO:teuthology.orchestra.run.smithi057.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-12T02:52:31.880 INFO:teuthology.orchestra.run.smithi057.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.8 MB/s | 4.9 MB 00:00 2024-09-12T02:52:31.914 INFO:teuthology.orchestra.run.smithi057.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-09-12T02:52:31.939 INFO:teuthology.orchestra.run.smithi057.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T02:52:31.998 INFO:teuthology.orchestra.run.smithi057.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T02:52:32.056 INFO:teuthology.orchestra.run.smithi057.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T02:52:32.118 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2024-09-12T02:52:32.118 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2024-09-12T02:52:32.132 INFO:teuthology.orchestra.run.smithi057.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-12T02:52:32.190 INFO:teuthology.orchestra.run.smithi057.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-09-12T02:52:32.241 INFO:teuthology.orchestra.run.smithi057.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-12T02:52:32.324 INFO:teuthology.orchestra.run.smithi057.stdout:(79/156): postgresql-server-13.16-1.el9.x86_64. 9.3 MB/s | 5.8 MB 00:00 2024-09-12T02:52:32.381 INFO:teuthology.orchestra.run.smithi057.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-12T02:52:32.400 INFO:teuthology.orchestra.run.smithi057.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-09-12T02:52:32.475 INFO:teuthology.orchestra.run.smithi057.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-12T02:52:32.500 INFO:teuthology.orchestra.run.smithi057.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-12T02:52:32.534 INFO:teuthology.orchestra.run.smithi057.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T02:52:32.559 INFO:teuthology.orchestra.run.smithi057.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T02:52:32.593 INFO:teuthology.orchestra.run.smithi057.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-09-12T02:52:32.618 INFO:teuthology.orchestra.run.smithi057.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T02:52:32.652 INFO:teuthology.orchestra.run.smithi057.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T02:52:32.677 INFO:teuthology.orchestra.run.smithi057.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-12T02:52:32.711 INFO:teuthology.orchestra.run.smithi057.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-12T02:52:32.736 INFO:teuthology.orchestra.run.smithi057.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-09-12T02:52:32.811 INFO:teuthology.orchestra.run.smithi057.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T02:52:32.837 INFO:teuthology.orchestra.run.smithi057.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-12T02:52:32.845 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-12T02:52:32.870 INFO:teuthology.orchestra.run.smithi057.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-12T02:52:32.895 INFO:teuthology.orchestra.run.smithi057.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:52:32.929 INFO:teuthology.orchestra.run.smithi057.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-12T02:52:32.954 INFO:teuthology.orchestra.run.smithi057.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-12T02:52:32.999 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-12T02:52:32.999 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-12T02:52:33.018 INFO:teuthology.orchestra.run.smithi057.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.3 MB/s | 304 kB 00:00 2024-09-12T02:52:33.046 INFO:teuthology.orchestra.run.smithi057.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 631 kB/s | 58 kB 00:00 2024-09-12T02:52:33.080 INFO:teuthology.orchestra.run.smithi057.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.5 MB/s | 96 kB 00:00 2024-09-12T02:52:33.105 INFO:teuthology.orchestra.run.smithi057.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T02:52:33.139 INFO:teuthology.orchestra.run.smithi057.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-09-12T02:52:33.447 INFO:teuthology.orchestra.run.smithi057.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-12T02:52:33.514 INFO:teuthology.orchestra.run.smithi057.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.6 MB/s | 548 kB 00:00 2024-09-12T02:52:33.548 INFO:teuthology.orchestra.run.smithi057.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T02:52:33.581 INFO:teuthology.orchestra.run.smithi057.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 907 kB/s | 19 kB 00:00 2024-09-12T02:52:33.698 INFO:teuthology.orchestra.run.smithi057.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-12T02:52:33.724 INFO:teuthology.orchestra.run.smithi057.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-12T02:52:33.749 INFO:teuthology.orchestra.run.smithi057.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:52:33.774 INFO:teuthology.orchestra.run.smithi057.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T02:52:33.800 INFO:teuthology.orchestra.run.smithi057.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T02:52:33.825 INFO:teuthology.orchestra.run.smithi057.stdout:(112/156): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-09-12T02:52:33.883 INFO:teuthology.orchestra.run.smithi057.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-09-12T02:52:33.909 INFO:teuthology.orchestra.run.smithi057.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-12T02:52:33.934 INFO:teuthology.orchestra.run.smithi057.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:52:33.959 INFO:teuthology.orchestra.run.smithi057.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-09-12T02:52:33.992 INFO:teuthology.orchestra.run.smithi057.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.2 MB/s | 172 kB 00:00 2024-09-12T02:52:34.025 INFO:teuthology.orchestra.run.smithi057.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-09-12T02:52:34.059 INFO:teuthology.orchestra.run.smithi057.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-12T02:52:34.084 INFO:teuthology.orchestra.run.smithi057.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T02:52:34.118 INFO:teuthology.orchestra.run.smithi057.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T02:52:34.151 INFO:teuthology.orchestra.run.smithi057.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T02:52:34.176 INFO:teuthology.orchestra.run.smithi057.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T02:52:34.202 INFO:teuthology.orchestra.run.smithi057.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-12T02:52:34.227 INFO:teuthology.orchestra.run.smithi057.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-12T02:52:34.252 INFO:teuthology.orchestra.run.smithi057.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-12T02:52:34.301 INFO:teuthology.orchestra.run.smithi057.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 412 kB/s | 20 kB 00:00 2024-09-12T02:52:34.361 INFO:teuthology.orchestra.run.smithi057.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-09-12T02:52:34.386 INFO:teuthology.orchestra.run.smithi057.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:52:34.428 INFO:teuthology.orchestra.run.smithi057.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.0 MB/s | 1.0 MB 00:00 2024-09-12T02:52:34.454 INFO:teuthology.orchestra.run.smithi057.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-09-12T02:52:34.471 INFO:teuthology.orchestra.run.smithi057.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-12T02:52:34.488 INFO:teuthology.orchestra.run.smithi057.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 987 kB/s | 16 kB 00:00 2024-09-12T02:52:34.513 INFO:teuthology.orchestra.run.smithi057.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T02:52:34.538 INFO:teuthology.orchestra.run.smithi057.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T02:52:34.564 INFO:teuthology.orchestra.run.smithi057.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-12T02:52:34.589 INFO:teuthology.orchestra.run.smithi057.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-12T02:52:34.614 INFO:teuthology.orchestra.run.smithi057.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:52:34.648 INFO:teuthology.orchestra.run.smithi057.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-12T02:52:34.673 INFO:teuthology.orchestra.run.smithi057.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-12T02:52:34.698 INFO:teuthology.orchestra.run.smithi057.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T02:52:34.724 INFO:teuthology.orchestra.run.smithi057.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:52:34.749 INFO:teuthology.orchestra.run.smithi057.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T02:52:34.774 INFO:teuthology.orchestra.run.smithi057.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-12T02:52:34.816 INFO:teuthology.orchestra.run.smithi057.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-12T02:52:34.826 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-12T02:52:34.826 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-12T02:52:34.841 INFO:teuthology.orchestra.run.smithi057.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-12T02:52:34.900 INFO:teuthology.orchestra.run.smithi057.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T02:52:35.117 INFO:teuthology.orchestra.run.smithi057.stdout:(148/156): protobuf-compiler-3.14.0-13.el9.x86_ 436 kB/s | 863 kB 00:01 2024-09-12T02:52:35.192 INFO:teuthology.orchestra.run.smithi057.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-12T02:52:35.259 INFO:teuthology.orchestra.run.smithi057.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-12T02:52:35.368 INFO:teuthology.orchestra.run.smithi057.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 377 kB/s | 41 kB 00:00 2024-09-12T02:52:35.476 INFO:teuthology.orchestra.run.smithi057.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-12T02:52:35.572 INFO:teuthology.orchestra.run.smithi057.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.8 MB/s | 1.8 MB 00:00 2024-09-12T02:52:35.619 INFO:teuthology.orchestra.run.smithi057.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 4.6 MB/s | 2.3 MB 00:00 2024-09-12T02:52:35.794 INFO:teuthology.orchestra.run.smithi057.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 1.4 MB 00:00 2024-09-12T02:52:35.927 INFO:teuthology.orchestra.run.smithi057.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-09-12T02:52:35.933 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:52:35.934 INFO:teuthology.orchestra.run.smithi057.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-12T02:52:38.010 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction check 2024-09-12T02:52:38.210 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction check succeeded. 2024-09-12T02:52:38.210 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction test 2024-09-12T02:52:38.482 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2024-09-12T02:52:38.595 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:52:38.633 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:52:38.686 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:52:38.729 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:52:38.754 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:52:38.797 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:52:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:52:39.230 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:39.353 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:39.381 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:39.473 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:39.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:52:39.651 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:52:39.708 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:52:39.735 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:39.831 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:39.861 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:39.907 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:39.929 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:39.955 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:40.121 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:40.251 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:40.378 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:52:40.477 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:40.557 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:40.694 INFO:teuthology.orchestra.run.smithi057.stdout:Transaction test succeeded. 2024-09-12T02:52:40.694 INFO:teuthology.orchestra.run.smithi057.stdout:Running transaction 2024-09-12T02:52:40.791 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:40.902 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:52:40.993 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:52:41.021 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:52:41.065 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:52:41.155 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:52:41.198 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:52:41.230 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:52:41.274 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:52:41.306 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:52:41.318 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-12T02:52:41.334 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:52:41.404 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:52:41.443 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:52:41.456 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:52:41.482 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:52:41.495 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:52:41.535 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:52:41.539 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:52:41.569 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:52:41.595 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:52:41.617 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:52:41.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:52:41.650 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:52:41.693 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:52:41.722 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:52:41.791 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:52:41.836 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:52:41.927 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:52:41.975 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:52:42.068 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:52:42.096 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:42.123 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:52:42.201 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:42.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:42.329 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:42.359 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:52:42.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:52:42.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:52:42.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:52:42.697 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:42.719 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:52:42.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:52:42.781 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:42.803 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:42.815 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:52:42.822 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:42.879 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:52:42.921 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:52:43.013 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:43.077 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:52:43.132 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:52:43.167 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:43.187 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:52:43.260 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:52:43.294 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:52:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:52:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:52:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:52:43.389 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:52:43.445 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:43.462 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:52:43.504 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:52:43.640 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:52:43.663 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:43.689 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:52:43.770 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:52:43.864 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:52:43.900 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:52:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:52:43.935 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:52:44.012 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:52:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:52:44.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:52:44.097 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:52:44.097 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:52:44.136 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:52:44.141 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:52:44.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:52:44.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:52:44.187 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:52:44.201 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:52:44.263 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:52:44.270 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:52:44.323 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:52:44.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:52:44.388 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:52:44.405 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:52:44.420 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:52:44.464 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:52:44.481 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:52:44.498 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:52:44.516 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:52:44.537 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:52:44.560 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:52:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:52:44.589 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:52:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:52:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:52:44.666 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:52:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:52:44.710 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:52:44.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:52:44.773 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:52:44.786 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:52:44.856 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:52:44.909 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:52:44.924 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:52:44.948 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:52:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:52:45.052 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:52:45.099 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:52:45.501 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:52:45.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:52:45.630 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:52:45.665 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:52:45.704 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:52:45.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:52:45.814 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:52:45.863 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:52:45.897 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:52:45.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:52:45.975 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:52:46.006 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:52:46.030 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:52:46.060 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:52:46.103 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:52:46.105 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:52:46.135 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:52:46.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:52:46.184 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:52:46.218 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:52:46.224 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:52:46.262 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:52:46.314 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:52:46.356 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:52:46.408 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:52:46.439 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:52:46.491 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:52:46.524 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:52:46.533 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:52:46.575 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:52:46.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:52:46.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:52:46.749 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:52:46.775 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:52:46.800 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:52:46.937 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:52:47.166 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:52:47.220 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:52:47.266 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:52:47.278 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:52:47.294 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:52:47.337 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:52:47.384 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:52:47.416 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:52:47.446 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:52:47.480 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:52:47.491 INFO:teuthology.orchestra.run.smithi057.stdout: Preparing : 1/1 2024-09-12T02:52:47.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:52:47.555 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:52:47.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:52:47.649 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:52:47.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:52:47.729 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:52:47.768 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:52:47.834 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:52:47.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:52:47.920 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:52:47.927 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:52:47.960 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:52:47.991 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:52:48.052 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:52:48.436 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:52:48.492 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:52:48.550 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:48.568 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:52:48.596 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:52:48.629 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:52:48.638 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:52:48.692 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:52:48.707 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:52:48.734 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:52:48.734 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:48.831 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:52:48.840 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:52:48.873 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:52:48.889 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:52:48.933 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:52:48.938 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:52:48.984 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:52:49.023 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:52:49.053 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:52:49.067 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:52:49.097 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:52:49.099 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:52:49.133 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:52:49.171 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:52:49.196 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:49.245 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:52:49.285 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:52:49.325 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:52:49.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:52:49.372 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:49.472 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:52:49.498 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:49.539 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:52:49.614 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:52:49.662 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:52:49.733 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:52:49.763 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:52:49.783 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:49.958 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:52:50.023 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:52:50.113 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:52:50.140 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:52:50.246 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:50.380 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:50.682 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:52:50.829 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:52:50.979 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:52:51.021 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:52:51.066 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:52:51.150 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:52:51.210 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:52:51.249 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:52:51.304 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:52:51.342 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:52:51.351 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:52:51.375 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:52:51.431 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:52:51.459 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:52:51.470 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:52:51.492 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:52:51.801 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:52:51.819 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:52:51.847 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:52:51.854 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:52:51.901 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:52:51.938 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:52:51.969 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:52:52.001 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:52:52.012 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:52:52.049 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:52:52.078 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:52.097 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:52:52.129 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:52:52.225 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:52:52.282 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:52:52.302 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:52.376 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:52:52.499 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:52:52.525 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:52:52.572 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:52:52.596 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:52:52.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:52:52.636 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:52:52.848 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:52.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:52:52.986 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:52:52.986 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:52:52.986 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:52:52.986 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:52:53.015 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:52:53.321 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:52:53.421 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:52:53.471 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:52:53.514 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:52:53.553 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:52:53.606 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:52:53.794 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:52:53.848 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:52:53.927 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:52:54.021 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:52:54.073 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:52:54.121 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:52:54.189 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:52:54.229 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:52:54.275 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:52:54.325 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:52:54.507 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:52:54.549 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:52:54.593 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:52:54.725 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:52:54.774 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:52:54.833 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:52:54.863 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:52:54.887 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:52:54.913 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:54.948 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:55.001 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:52:55.196 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:52:55.237 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:52:55.274 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:52:55.311 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:52:55.357 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:52:55.396 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:52:55.433 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:52:55.476 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:52:55.494 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:52:55.509 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:52:55.535 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:52:55.569 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:52:55.601 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:52:55.626 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:52:55.627 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:52:55.627 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:52:55.627 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:52:55.653 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:52:55.704 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:52:55.760 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:52:55.796 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:52:55.927 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:52:55.993 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:52:56.930 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:52:56.986 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:52:57.020 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:52:57.138 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:52:57.195 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:52:57.266 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:52:57.381 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:52:57.422 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:52:57.670 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:52:57.710 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:52:57.808 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:52:58.667 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:52:58.822 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:53:00.507 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:53:00.766 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:53:00.828 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:53:00.861 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:53:01.213 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:53:01.263 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:53:01.426 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:53:01.461 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:53:01.510 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:53:02.071 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:53:02.108 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:53:02.199 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:53:02.226 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:53:04.684 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2024-09-12T02:53:04.685 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:04.868 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:04.885 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:04.885 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:04.885 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:53:04.885 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:05.054 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:53:05.105 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:53:05.772 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:53:05.817 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:53:05.966 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:53:05.966 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:53:05.966 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:53:05.966 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:06.010 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:53:06.379 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:06.407 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:06.639 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:06.663 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:07.123 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:07.146 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:07.147 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:07.147 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:53:07.147 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:07.147 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:07.147 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:07.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:53:07.318 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:53:07.373 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-09-12T02:53:07.489 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-09-12T02:53:07.490 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:07.558 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:53:07.615 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:53:07.686 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:07.692 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:53:07.703 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:07.703 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:07.703 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:53:07.703 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:07.790 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:53:07.929 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:53:07.982 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:53:08.014 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:53:08.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:53:08.426 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:08.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:08.488 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:53:08.550 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:53:08.622 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:09.195 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:09.222 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:09.454 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:09.478 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:09.677 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:09.732 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:09.921 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:09.945 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:10.060 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:53:10.116 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:53:10.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:53:10.365 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:53:10.414 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:53:10.491 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:53:10.581 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:53:10.703 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:10.710 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:53:10.732 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:10.764 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:53:10.796 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:53:10.824 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:53:10.836 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:11.004 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:53:11.034 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:11.058 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:11.058 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:11.058 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:53:11.058 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:11.058 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:11.059 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:11.110 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:11.122 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:11.207 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:11.220 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:53:11.256 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:11.275 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:53:11.296 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:53:11.360 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:53:11.396 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:12.199 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:53:12.229 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:12.254 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:12.305 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:12.328 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:12.328 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:12.328 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:53:12.329 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:12.397 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:12.420 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:12.421 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:12.421 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:53:12.421 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:12.421 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:12.421 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:12.441 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:12.490 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:12.723 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:12.746 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:12.746 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:12.746 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:53:12.746 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:12.746 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:12.747 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:13.454 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:13.482 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:13.584 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:13.754 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:53:13.784 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:13.808 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:13.860 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:13.872 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:13.969 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:53:14.024 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:53:14.946 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:53:14.987 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:15.011 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:15.012 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:15.012 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:53:15.012 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:15.012 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:15.012 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:15.055 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:15.078 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:15.078 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:15.078 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:53:15.079 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:15.130 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:15.154 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:15.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:15.480 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:17.318 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:53:17.359 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:53:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:53:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:53:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:53:17.664 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:53:17.714 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:53:17.745 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:53:17.774 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:53:17.821 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:53:18.445 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:18.485 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:18.506 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /sys 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /proc 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /mnt 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /var/tmp 2024-09-12T02:53:19.328 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /home 2024-09-12T02:53:19.329 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /root 2024-09-12T02:53:19.329 INFO:teuthology.orchestra.run.smithi057.stdout:skipping the directory /tmp 2024-09-12T02:53:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:19.349 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:53:19.449 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:53:19.489 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:53:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:19.572 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:53:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:53:19.579 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:19.579 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:53:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:19.610 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:53:19.635 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:53:19.661 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:53:19.662 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:19.762 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:19.762 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:19.846 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:19.871 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:53:19.877 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:19.877 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:20.019 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:53:20.060 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:53:20.065 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:20.065 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:20.109 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:53:20.150 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:53:20.165 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:20.177 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:53:20.229 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:20.358 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:53:20.399 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:53:20.429 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:53:20.459 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:53:20.505 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:53:21.123 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:21.161 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:21.182 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:21.256 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:21.314 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:53:21.315 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:21.315 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:53:21.315 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:21.315 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:53:21.315 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:21.573 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:53:21.625 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:22.022 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:53:22.108 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:53:22.132 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:22.149 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:53:22.183 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:22.230 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:53:22.262 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:53:22.287 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:53:22.313 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:53:22.313 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:22.325 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:53:22.397 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:22.397 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:22.402 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:53:22.476 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:53:22.489 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:22.514 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:53:22.520 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:22.520 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:22.700 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:22.700 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:22.713 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:53:22.771 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:53:22.792 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:22.865 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:22.867 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:53:22.979 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:53:23.174 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:53:23.227 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:53:23.275 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:53:23.309 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:53:23.905 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:23.933 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:53:23.975 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:53:24.062 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:53:24.097 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:25.400 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:53:25.441 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:53:25.838 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:53:25.839 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:53:25.840 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:53:25.852 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:53:25.853 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:53:25.854 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:53:25.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:53:25.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:53:25.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:53:25.858 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:53:25.859 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:53:25.860 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:53:25.861 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:53:25.862 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:53:26.595 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:53:26.624 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:26.769 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:53:26.956 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:53:26.985 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:27.036 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:53:27.037 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:27.037 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:53:27.037 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:27.037 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:53:27.037 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:27.094 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:27.121 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:53:27.240 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:53:27.312 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:53:28.372 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:53:28.404 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:28.455 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:53:28.456 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:28.456 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:53:28.456 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:28.456 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:53:28.456 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:28.505 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:28.555 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:53:28.556 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:28.556 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:53:28.556 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:28.606 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:28.657 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:53:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:53:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:53:28.658 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:28.753 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:53:28.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:53:28.755 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:53:28.756 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:53:28.756 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:53:28.756 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:53:28.756 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:53:28.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:53:28.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:53:28.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:53:28.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:53:28.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:53:28.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:53:28.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:53:28.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:53:28.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:53:28.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:53:28.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:53:28.775 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:53:28.776 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:53:28.777 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.784 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.785 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.786 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:53:28.787 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:53:28.788 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:53:28.789 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:53:28.790 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:53:28.791 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:53:28.792 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:53:28.793 INFO:teuthology.orchestra.run.smithi181.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.794 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:53:28.795 INFO:teuthology.orchestra.run.smithi181.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:53:28.796 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:53:28.797 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:53:28.797 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:53:28.797 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2024-09-12T02:53:29.014 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:53:29.066 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:29.234 DEBUG:teuthology.parallel:result is None 2024-09-12T02:53:31.720 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-12T02:53:31.721 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.722 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.723 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.724 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:53:31.725 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:53:31.726 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:53:31.727 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:53:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:53:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:53:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:53:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:53:31.732 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:53:31.733 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:53:31.733 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:53:31.733 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-12T02:53:32.180 DEBUG:teuthology.parallel:result is None 2024-09-12T02:53:34.320 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:53:34.370 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:53:34.414 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:53:34.451 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:53:34.486 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:53:34.762 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:53:34.816 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:53:34.845 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:53:34.877 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:53:34.946 INFO:teuthology.orchestra.run.smithi057.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:53:35.708 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:35.760 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:35.802 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:53:37.173 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:53:37.282 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:53:37.345 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:53:37.459 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:53:37.500 INFO:teuthology.orchestra.run.smithi057.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:53:37.532 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:53:37.579 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:53:37.579 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:37.720 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:53:37.720 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:37.829 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:53:37.853 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:53:37.870 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:37.870 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:38.130 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:53:38.130 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:38.248 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:53:38.326 INFO:teuthology.orchestra.run.smithi057.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:38.516 INFO:teuthology.orchestra.run.smithi057.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:38.517 INFO:teuthology.orchestra.run.smithi057.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-12T02:53:38.517 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:53:44.096 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:53:44.097 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:53:44.098 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:53:44.110 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:53:44.111 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:53:44.112 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:53:44.113 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:53:44.114 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:53:44.115 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:53:44.116 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:53:44.117 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:53:44.118 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:53:44.119 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:53:44.120 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:53:46.304 INFO:teuthology.orchestra.run.smithi057.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:53:46.304 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:46.304 INFO:teuthology.orchestra.run.smithi057.stdout:Upgraded: 2024-09-12T02:53:46.304 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout:Installed: 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:53:46.305 INFO:teuthology.orchestra.run.smithi057.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.306 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:53:46.307 INFO:teuthology.orchestra.run.smithi057.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.308 INFO:teuthology.orchestra.run.smithi057.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:53:46.309 INFO:teuthology.orchestra.run.smithi057.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:53:46.310 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:53:46.311 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:53:46.312 INFO:teuthology.orchestra.run.smithi057.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:53:46.313 INFO:teuthology.orchestra.run.smithi057.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:53:46.314 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:53:46.315 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:53:46.316 INFO:teuthology.orchestra.run.smithi057.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:53:46.317 INFO:teuthology.orchestra.run.smithi057.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:53:46.317 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:53:46.317 INFO:teuthology.orchestra.run.smithi057.stdout:Complete! 2024-09-12T02:53:46.959 DEBUG:teuthology.parallel:result is None 2024-09-12T02:53:46.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:47.106 DEBUG:teuthology.orchestra.run.smithi057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:53:47.152 INFO:teuthology.orchestra.run.smithi057.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.152 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.153 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:53:47.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:47.295 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:53:47.322 INFO:teuthology.orchestra.run.smithi181.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.323 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.323 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:53:47.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:47.466 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:53:47.493 INFO:teuthology.orchestra.run.smithi204.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.493 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:53:47.493 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:53:47.495 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T02:53:47.495 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:53:47.495 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:53:47.554 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:53:47.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:53:47.587 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:53:47.587 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:53:47.620 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T02:53:47.620 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:53:47.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:53:47.676 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:53:47.774 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:53:47.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:53:47.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:53:47.869 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:53:47.870 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:53:47.901 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:53:47.969 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T02:53:47.969 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:53:47.969 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:53:48.026 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:53:48.122 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:53:48.122 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:53:48.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:53:48.216 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:53:48.216 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:53:48.241 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:53:48.307 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T02:53:48.307 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:53:48.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:53:48.364 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:53:48.460 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:53:48.461 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:53:48.486 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:53:48.550 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:53:48.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:53:48.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:53:48.641 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T02:53:48.737 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:53:48.737 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:48.737 INFO:tasks.cephadm:Cluster fsid is 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:53:48.738 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T02:53:48.738 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.57', 'mon.b': '172.21.15.181', 'mon.c': '172.21.15.204'} 2024-09-12T02:53:48.738 INFO:tasks.cephadm:First mon is mon.a on smithi057 2024-09-12T02:53:48.738 INFO:tasks.cephadm:First mgr is x 2024-09-12T02:53:48.738 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T02:53:48.738 DEBUG:teuthology.orchestra.run.smithi057:> sudo hostname $(hostname -s) 2024-09-12T02:53:48.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname $(hostname -s) 2024-09-12T02:53:48.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-09-12T02:53:48.852 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T02:53:48.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:49.003 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T02:53:49.152 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:53:49.153 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:53:49.154 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:53:49.154 DEBUG:teuthology.orchestra.run.smithi057:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:49.584 INFO:teuthology.orchestra.run.smithi057.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:53 /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:49.584 DEBUG:teuthology.orchestra.run.smithi181:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:49.976 INFO:teuthology.orchestra.run.smithi181.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:53 /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:49.977 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:50.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:53 /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:50.376 DEBUG:teuthology.orchestra.run.smithi057:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:50.413 DEBUG:teuthology.orchestra.run.smithi181:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:50.433 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:53:50.467 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T02:53:50.467 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:53:50.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:53:50.476 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:53:52.083 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:53:52.094 INFO:teuthology.orchestra.run.smithi181.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:53:52.492 INFO:teuthology.orchestra.run.smithi057.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:54:20.904 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-09-12T02:54:20.905 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout: "repo_digests": [ 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T02:54:22.483 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout: "repo_digests": [ 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-12T02:54:24.212 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-12T02:54:24.236 DEBUG:teuthology.orchestra.run.smithi057:> sudo mkdir -p /etc/ceph 2024-09-12T02:54:24.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph 2024-09-12T02:54:24.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-09-12T02:54:24.358 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 777 /etc/ceph 2024-09-12T02:54:24.418 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /etc/ceph 2024-09-12T02:54:24.442 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-09-12T02:54:24.472 INFO:tasks.cephadm:Writing seed config... 2024-09-12T02:54:24.474 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T02:54:24.474 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T02:54:24.474 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T02:54:24.474 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T02:54:24.475 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T02:54:24.475 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T02:54:24.475 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T02:54:24.475 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T02:54:24.475 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T02:54:24.476 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T02:54:24.477 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T02:54:24.477 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T02:54:24.477 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T02:54:24.477 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T02:54:24.477 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T02:54:24.478 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T02:54:24.479 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T02:54:24.480 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T02:54:24.481 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-12T02:54:24.482 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:54:24.482 DEBUG:teuthology.orchestra.run.smithi057:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T02:54:24.520 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3c25d9c4-70b2-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T02:54:24.520 DEBUG:teuthology.orchestra.run.smithi057:mon.a> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T02:54:24.564 DEBUG:teuthology.orchestra.run.smithi057:mgr.x> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T02:54:24.609 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T02:54:24.610 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.57 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:24.872 INFO:teuthology.orchestra.run.smithi057.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:54:24.872 INFO:teuthology.orchestra.run.smithi057.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '3c25d9c4-70b2-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.57', '--skip-admin-label'] 2024-09-12T02:54:24.872 INFO:teuthology.orchestra.run.smithi057.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T02:54:24.873 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying podman|docker is present... 2024-09-12T02:54:24.919 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:54:24.920 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying lvm2 is present... 2024-09-12T02:54:24.920 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying time synchronization is in place... 2024-09-12T02:54:24.934 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:54:24.934 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:54:24.947 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:54:24.947 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:54:24.961 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-09-12T02:54:24.974 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-09-12T02:54:24.974 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:54:24.974 INFO:teuthology.orchestra.run.smithi057.stdout:Repeating the final host check... 2024-09-12T02:54:25.018 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:54:25.018 INFO:teuthology.orchestra.run.smithi057.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T02:54:25.018 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl is present 2024-09-12T02:54:25.018 INFO:teuthology.orchestra.run.smithi057.stdout:lvcreate is present 2024-09-12T02:54:25.030 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:54:25.031 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:54:25.044 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:54:25.045 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:54:25.061 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout enabled 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout active 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:Host looks OK 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:Cluster fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:Acquiring lock 139864017724272 on /run/cephadm/3c25d9c4-70b2-11ef-bcea-c7b262605968.lock 2024-09-12T02:54:25.075 INFO:teuthology.orchestra.run.smithi057.stdout:Lock 139864017724272 acquired on /run/cephadm/3c25d9c4-70b2-11ef-bcea-c7b262605968.lock 2024-09-12T02:54:25.076 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 3300 ... 2024-09-12T02:54:25.076 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying IP 172.21.15.57 port 6789 ... 2024-09-12T02:54:25.076 INFO:teuthology.orchestra.run.smithi057.stdout:Base mon IP(s) is [172.21.15.57:3300, 172.21.15.57:6789], mon addrv is [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-09-12T02:54:25.083 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.57 metric 100 2024-09-12T02:54:25.083 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.57 metric 100 2024-09-12T02:54:25.089 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T02:54:25.089 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T02:54:25.089 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1562/64 scope link 2024-09-12T02:54:25.095 INFO:teuthology.orchestra.run.smithi057.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:54:25.096 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-09-12T02:54:25.096 INFO:teuthology.orchestra.run.smithi057.stdout:Mon IP `172.21.15.57` is in CIDR network `172.21.0.0/20` 2024-09-12T02:54:25.096 INFO:teuthology.orchestra.run.smithi057.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T02:54:25.097 INFO:teuthology.orchestra.run.smithi057.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T02:54:25.098 INFO:teuthology.orchestra.run.smithi057.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:54:25.595 INFO:teuthology.orchestra.run.smithi057.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T02:54:26.694 INFO:teuthology.orchestra.run.smithi057.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:54:26.695 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:54:26.695 INFO:teuthology.orchestra.run.smithi057.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T02:54:37.334 INFO:teuthology.orchestra.run.smithi057.stdout:stat: stdout 167 167 2024-09-12T02:54:37.334 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial keys... 2024-09-12T02:54:38.422 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQDtV+JmNhE3KhAAXXYy6ZdFLjAr8lq84XK0yQ== 2024-09-12T02:54:39.421 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQDuV+JmtHmGLxAAf13iVSxpEVVtaPDGyXIEfA== 2024-09-12T02:54:40.459 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph-authtool: stdout AQDvV+JmXn7SLxAAr6crKQ1IvlZ1p+6aYW6UUw== 2024-09-12T02:54:40.460 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial monmap... 2024-09-12T02:54:41.492 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:54:41.492 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T02:54:41.492 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:41.492 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout:monmaptool for a [v2:172.21.15.57:3300,v1:172.21.15.57:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout:setting min_mon_release = quincy 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: set fsid to 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:54:41.493 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mon... 2024-09-12T02:54:43.350 INFO:teuthology.orchestra.run.smithi057.stdout:create mon.a on 2024-09-12T02:54:43.691 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T02:54:43.909 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T02:54:44.176 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target. 2024-09-12T02:54:44.176 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target. 2024-09-12T02:54:44.513 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:54:44.513 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T02:54:44.782 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target.wants/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@.service. 2024-09-12T02:54:45.439 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:45 smithi057 podman[29986]: 2024-09-12 02:54:45.379332725 +0000 UTC m=+0.360033227 container start 2a6e60341725fd81dd5fcd7673313091b7f7a03a7f5e61bbb08ae049d91638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-12T02:54:45.467 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:54:45.467 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-09-12T02:54:45.480 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:54:45.480 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:54:45.481 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-09-12T02:54:45.481 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:54:45.481 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon to start... 2024-09-12T02:54:45.481 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mon... 2024-09-12T02:54:45.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:45 smithi057 bash[29986]: 2a6e60341725fd81dd5fcd7673313091b7f7a03a7f5e61bbb08ae049d91638ed 2024-09-12T02:54:45.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:45 smithi057 systemd[1]: Started Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:54:45.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:45 smithi057 ceph-mon[30004]: mkfs 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:45.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:45 smithi057 ceph-mon[30004]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: monmap epoch 1 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: last_changed 2024-09-12T02:54:40.839595+0000 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: min_mon_release 19 (squid) 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: election_strategy: 1 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:54:46.696 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: fsmap 2024-09-12T02:54:46.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:54:46.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: mgrmap e1: no daemons active 2024-09-12T02:54:46.697 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:46 smithi057 ceph-mon[30004]: from='client.? 172.21.15.57:0/987425245' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T02:54:46.703 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout id: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout services: 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.648808s) 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout data: 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T02:54:46.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.017+0000 7f3341e00640 1 Processor -- start 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.017+0000 7f3341e00640 1 -- start start 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.017+0000 7f3341e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.017+0000 7f3341e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f333c07c600 con 0x7f333c108190 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.018+0000 7f333aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.018+0000 7f333aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37860/0 (socket says 172.21.15.57:37860) 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.018+0000 7f333aa00640 1 -- 172.21.15.57:0/3849962674 learned_addr learned my addr 172.21.15.57:0/3849962674 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:46.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.019+0000 7f333aa00640 1 -- 172.21.15.57:0/3849962674 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c10aac0 con 0x7f333c108190 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.021+0000 7f333aa00640 1 --2- 172.21.15.57:0/3849962674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f332c009820 tx=0x7f332c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58c0b95a4900b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3339600640 1 -- 172.21.15.57:0/3849962674 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c037070 con 0x7f333c108190 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3339600640 1 -- 172.21.15.57:0/3849962674 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c032440 con 0x7f333c108190 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3339600640 1 -- 172.21.15.57:0/3849962674 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c032740 con 0x7f333c108190 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 -- 172.21.15.57:0/3849962674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 msgr2=0x7f333c10a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 --2- 172.21.15.57:0/3849962674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f332c009820 tx=0x7f332c02a270 comp rx=0 tx=0).stop 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 -- 172.21.15.57:0/3849962674 shutdown_connections 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 --2- 172.21.15.57:0/3849962674 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c10a580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 -- 172.21.15.57:0/3849962674 >> 172.21.15.57:0/3849962674 conn(0x7f333c101d10 msgr2=0x7f333c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.022+0000 7f3341e00640 1 -- 172.21.15.57:0/3849962674 shutdown_connections 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.023+0000 7f3341e00640 1 -- 172.21.15.57:0/3849962674 wait complete. 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f3341e00640 1 Processor -- start 2024-09-12T02:54:46.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f3341e00640 1 -- start start 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f3341e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f3341e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10af40 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f333aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f333aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37864/0 (socket says 172.21.15.57:37864) 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.024+0000 7f333aa00640 1 -- 172.21.15.57:0/987425245 learned_addr learned my addr 172.21.15.57:0/987425245 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f333aa00640 1 -- 172.21.15.57:0/987425245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c107f50 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f333aa00640 1 --2- 172.21.15.57:0/987425245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f332c0090b0 tx=0x7f332c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c041070 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c03c7a0 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c037040 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.025+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c106140 con 0x7f333c108190 2024-09-12T02:54:46.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.026+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c106680 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.026+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f332c044070 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.026+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f332c03d3f0 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.027+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c1085f0 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.030+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f332c046450 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.095+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f333c194430 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.096+0000 7f3340a00640 1 -- 172.21.15.57:0/987425245 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f332c046630 con 0x7f333c108190 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 msgr2=0x7f333c107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 --2- 172.21.15.57:0/987425245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f332c0090b0 tx=0x7f332c0046c0 comp rx=0 tx=0).stop 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 shutdown_connections 2024-09-12T02:54:46.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 --2- 172.21.15.57:0/987425245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f333c108190 0x7f333c107a10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:46.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 >> 172.21.15.57:0/987425245 conn(0x7f333c101d10 msgr2=0x7f333c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:46.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 shutdown_connections 2024-09-12T02:54:46.714 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:46.097+0000 7f3341e00640 1 -- 172.21.15.57:0/987425245 wait complete. 2024-09-12T02:54:46.714 INFO:teuthology.orchestra.run.smithi057.stdout:mon is available 2024-09-12T02:54:46.714 INFO:teuthology.orchestra.run.smithi057.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T02:54:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:47 smithi057 ceph-mon[30004]: from='client.? 172.21.15.57:0/2067425054' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:54:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:47 smithi057 ceph-mon[30004]: from='client.? 172.21.15.57:0/2067425054' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T02:54:48.021 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:54:48.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c23200640 1 Processor -- start 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c23200640 1 -- start start 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c23200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c23200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c10a250 con 0x7f5c1c109880 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.231+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37878/0 (socket says 172.21.15.57:37878) 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.232+0000 7f5c1be00640 1 -- 172.21.15.57:0/3168836890 learned_addr learned my addr 172.21.15.57:0/3168836890 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.232+0000 7f5c1be00640 1 -- 172.21.15.57:0/3168836890 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c10aa80 con 0x7f5c1c109880 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.233+0000 7f5c1be00640 1 --2- 172.21.15.57:0/3168836890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009820 tx=0x7f5c0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c377f417e297ee48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.233+0000 7f5c1aa00640 1 -- 172.21.15.57:0/3168836890 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c037070 con 0x7f5c1c109880 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.233+0000 7f5c1aa00640 1 -- 172.21.15.57:0/3168836890 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c032440 con 0x7f5c1c109880 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.233+0000 7f5c1aa00640 1 -- 172.21.15.57:0/3168836890 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c032740 con 0x7f5c1c109880 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 -- 172.21.15.57:0/3168836890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 msgr2=0x7f5c1c109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 --2- 172.21.15.57:0/3168836890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009820 tx=0x7f5c0c02a270 comp rx=0 tx=0).stop 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 -- 172.21.15.57:0/3168836890 shutdown_connections 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 --2- 172.21.15.57:0/3168836890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c109c80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 -- 172.21.15.57:0/3168836890 >> 172.21.15.57:0/3168836890 conn(0x7f5c1c07ce60 msgr2=0x7f5c1c07d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 -- 172.21.15.57:0/3168836890 shutdown_connections 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.234+0000 7f5c23200640 1 -- 172.21.15.57:0/3168836890 wait complete. 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c23200640 1 Processor -- start 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c23200640 1 -- start start 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c23200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c23200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c10b020 con 0x7f5c1c109880 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c1be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37888/0 (socket says 172.21.15.57:37888) 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.235+0000 7f5c1be00640 1 -- 172.21.15.57:0/2067425054 learned_addr learned my addr 172.21.15.57:0/2067425054 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c1be00640 1 -- 172.21.15.57:0/2067425054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c19d580 con 0x7f5c1c109880 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c1be00640 1 --2- 172.21.15.57:0/2067425054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0090b0 tx=0x7f5c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c041070 con 0x7f5c1c109880 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c03c7a0 con 0x7f5c1c109880 2024-09-12T02:54:48.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c19d810 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.236+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c037040 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.237+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c1a0500 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.237+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c0c044070 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.237+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c0c03d730 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.238+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c10b670 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.240+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5c0c03cbe0 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.302+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5c1c1a1bd0 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.307+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f5c0c03cdc0 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.308+0000 7f5c18c00640 1 -- 172.21.15.57:0/2067425054 <== mon.0 v2:172.21.15.57:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c0c032440 con 0x7f5c1c109880 2024-09-12T02:54:48.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 msgr2=0x7f5c1c19d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 --2- 172.21.15.57:0/2067425054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0090b0 tx=0x7f5c0c0046c0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 shutdown_connections 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 --2- 172.21.15.57:0/2067425054 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c1c109880 0x7f5c1c19d040 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 >> 172.21.15.57:0/2067425054 conn(0x7f5c1c07ce60 msgr2=0x7f5c1c191330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 shutdown_connections 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:47.309+0000 7f5c23200640 1 -- 172.21.15.57:0/2067425054 wait complete. 2024-09-12T02:54:48.027 INFO:teuthology.orchestra.run.smithi057.stdout:Generating new minimal ceph.conf... 2024-09-12T02:54:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:48 smithi057 ceph-mon[30004]: from='client.? 172.21.15.57:0/1691961648' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.529+0000 7f4689400640 1 Processor -- start 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.529+0000 7f4689400640 1 -- start start 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.529+0000 7f4689400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.529+0000 7f4689400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f468410a250 con 0x7f4684109880 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.530+0000 7f4682800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.530+0000 7f4682800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51164/0 (socket says 172.21.15.57:51164) 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.530+0000 7f4682800640 1 -- 172.21.15.57:0/2402059570 learned_addr learned my addr 172.21.15.57:0/2402059570 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:49.213 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.530+0000 7f4682800640 1 -- 172.21.15.57:0/2402059570 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468410aa80 con 0x7f4684109880 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.531+0000 7f4682800640 1 --2- 172.21.15.57:0/2402059570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4674009820 tx=0x7f467402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e573c2248fd59d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.531+0000 7f4681400640 1 -- 172.21.15.57:0/2402059570 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674037070 con 0x7f4684109880 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.531+0000 7f4681400640 1 -- 172.21.15.57:0/2402059570 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4674032900 con 0x7f4684109880 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.531+0000 7f4681400640 1 -- 172.21.15.57:0/2402059570 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674030850 con 0x7f4684109880 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 -- 172.21.15.57:0/2402059570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 msgr2=0x7f4684109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 --2- 172.21.15.57:0/2402059570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4674009820 tx=0x7f467402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 -- 172.21.15.57:0/2402059570 shutdown_connections 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 --2- 172.21.15.57:0/2402059570 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f4684109c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 -- 172.21.15.57:0/2402059570 >> 172.21.15.57:0/2402059570 conn(0x7f468407ce60 msgr2=0x7f468407d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 -- 172.21.15.57:0/2402059570 shutdown_connections 2024-09-12T02:54:49.214 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.532+0000 7f4689400640 1 -- 172.21.15.57:0/2402059570 wait complete. 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.533+0000 7f4689400640 1 Processor -- start 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.533+0000 7f4689400640 1 -- start start 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4689400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4689400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f468410de30 con 0x7f4684109880 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4682800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4682800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51176/0 (socket says 172.21.15.57:51176) 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4682800640 1 -- 172.21.15.57:0/1691961648 learned_addr learned my addr 172.21.15.57:0/1691961648 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4682800640 1 -- 172.21.15.57:0/1691961648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468419ddc0 con 0x7f4684109880 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.534+0000 7f4682800640 1 --2- 172.21.15.57:0/1691961648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46740097f0 tx=0x7f4674030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4674037040 con 0x7f4684109880 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4674040070 con 0x7f4684109880 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468419e050 con 0x7f4684109880 2024-09-12T02:54:49.215 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467403a390 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46841a0d40 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f467403a5e0 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.535+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4674045950 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.536+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468410dfb0 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.539+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4674045460 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.601+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f46841a0f10 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.602+0000 7f467b400640 1 -- 172.21.15.57:0/1691961648 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f467403b430 con 0x7f4684109880 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 msgr2=0x7f468419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 --2- 172.21.15.57:0/1691961648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46740097f0 tx=0x7f4674030d80 comp rx=0 tx=0).stop 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 shutdown_connections 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 --2- 172.21.15.57:0/1691961648 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4684109880 0x7f468419d880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:49.216 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 >> 172.21.15.57:0/1691961648 conn(0x7f468407ce60 msgr2=0x7f4684106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:49.217 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 shutdown_connections 2024-09-12T02:54:49.217 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:48.603+0000 7f4689400640 1 -- 172.21.15.57:0/1691961648 wait complete. 2024-09-12T02:54:49.217 INFO:teuthology.orchestra.run.smithi057.stdout:Restarting the monitor... 2024-09-12T02:54:49.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:49 smithi057 systemd[1]: Stopping Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:54:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a[30000]: 2024-09-12T02:54:49.607+0000 7fd130c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:54:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a[30000]: 2024-09-12T02:54:49.607+0000 7fd130c00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T02:54:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:49 smithi057 podman[30243]: 2024-09-12 02:54:49.628820444 +0000 UTC m=+0.176990592 container died 2a6e60341725fd81dd5fcd7673313091b7f7a03a7f5e61bbb08ae049d91638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True) 2024-09-12T02:54:50.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 podman[30243]: 2024-09-12 02:54:50.004084326 +0000 UTC m=+0.552254510 container cleanup 2a6e60341725fd81dd5fcd7673313091b7f7a03a7f5e61bbb08ae049d91638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:54:50.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 bash[30243]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:54:50.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 podman[30255]: 2024-09-12 02:54:50.288698612 +0000 UTC m=+0.652305528 container remove 2a6e60341725fd81dd5fcd7673313091b7f7a03a7f5e61bbb08ae049d91638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:54:50.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:54:50.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 systemd[1]: Stopped Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:54:50.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.219s CPU time. 2024-09-12T02:54:50.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 systemd[1]: Starting Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:54:51.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 podman[30343]: 2024-09-12 02:54:50.712880889 +0000 UTC m=+0.027125684 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:54:51.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:50 smithi057 podman[30343]: 2024-09-12 02:54:50.881422922 +0000 UTC m=+0.195667714 container create 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T02:54:51.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 podman[30343]: 2024-09-12 02:54:51.084350996 +0000 UTC m=+0.398595791 container init 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T02:54:51.184 INFO:teuthology.orchestra.run.smithi057.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T02:54:51.337 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 podman[30343]: 2024-09-12 02:54:51.091940923 +0000 UTC m=+0.406185718 container start 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:54:51.337 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:54:51.337 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:54:51.337 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: pidfile_write: ignore empty --pid-file 2024-09-12T02:54:51.337 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: load: jerasure load: lrc 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Git sha 0 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: DB SUMMARY 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: DB Session ID: T2X9AZ1VWBAKU78QO4WI 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T02:54:51.338 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105832 ; 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.env: 0x5593250d2940 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.info_log: 0x55934c7ef6a0 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.statistics: (nil) 2024-09-12T02:54:51.339 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.use_fsync: 0 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:54:51.340 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.db_log_dir: 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.wal_dir: 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:54:51.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.write_buffer_manager: 0x55934c7fedc0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:54:51.343 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.unordered_write: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.row_cache: None 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.wal_filter: None 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.wal_compression: 0 2024-09-12T02:54:51.344 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:54:51.345 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_open_files: -1 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:54:51.346 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Compression algorithms supported: 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kZSTD supported: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:54:51.347 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.merge_operator: 2024-09-12T02:54:51.348 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_filter: None 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55934c7ef300) 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:54:51.349 INFO:journalctl@ceph.mon.a.smithi057.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_type: 0 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_index_type: 0 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_shortening: 1 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: checksum: 4 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: no_block_cache: 0 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache: 0x55934c7e31f0 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_options: 2024-09-12T02:54:51.350 INFO:journalctl@ceph.mon.a.smithi057.stdout: capacity : 536870912 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_shard_bits : 4 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: strict_capacity_limit : 0 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_cache_compressed: (nil) 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: persistent_cache: (nil) 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size: 4096 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_size_deviation: 10 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_restart_interval: 16 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: index_block_restart_interval: 1 2024-09-12T02:54:51.351 INFO:journalctl@ceph.mon.a.smithi057.stdout: metadata_block_size: 4096 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: partition_filters: 0 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: use_delta_encoding: 1 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: filter_policy: bloomfilter 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: whole_key_filtering: 1 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: verify_compression: 0 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: format_version: 5 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_index_compression: 1 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: block_align: 0 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: max_auto_readahead_size: 262144 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: prepopulate_block_cache: 0 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:54:51.352 INFO:journalctl@ceph.mon.a.smithi057.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression: NoCompression 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.num_levels: 7 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:54:51.353 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:54:51.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:54:51.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:54:51.356 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:54:51.357 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.ttl: 2592000 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:54:51.358 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2354717e-ab1e-4da5-a43e-1d4170f1616d 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109691133939, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T02:54:51.359 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109691135481, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98749, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14433, "raw_average_key_size": 49, "raw_value_size": 90924, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109691, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2354717e-ab1e-4da5-a43e-1d4170f1616d", "db_session_id": "T2X9AZ1VWBAKU78QO4WI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109691135654, "job": 1, "event": "recovery_finished"} 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55934c812e00 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: DB pointer 0x55934c90e000 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** DB Stats ** 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:51.360 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: L0 2/0 100.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Sum 2/0 100.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** Compaction Stats [default] ** 2024-09-12T02:54:51.361 INFO:journalctl@ceph.mon.a.smithi057.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 112.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Cumulative compaction: 0.00 GB write, 15.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Interval compaction: 0.00 GB write, 15.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:54:51.362 INFO:journalctl@ceph.mon.a.smithi057.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache BinnedLRUCache@0x55934c7e31f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: starting mon.a rank 0 at public addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] at bind addrs [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???) e1 preinit fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).mds e1 new map 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).mds e1 print_map 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: e1 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: btime 2024-09-12T02:54:45:449413+0000 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: legacy client fscid: -1 2024-09-12T02:54:51.363 INFO:journalctl@ceph.mon.a.smithi057.stdout: 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout: No filesystems configured 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 bash[30343]: 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 systemd[1]: Started Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: monmap epoch 1 2024-09-12T02:54:51.364 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: last_changed 2024-09-12T02:54:40.839595+0000 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: min_mon_release 19 (squid) 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: election_strategy: 1 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: fsmap 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:54:51.365 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:51 smithi057 ceph-mon[30361]: mgrmap e1: no daemons active 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.803+0000 7f91c8c00640 1 Processor -- start 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.803+0000 7f91c8c00640 1 -- start start 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.804+0000 7f91c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.804+0000 7f91c8c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f91c407c830 con 0x7f91c407de00 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.804+0000 7f91c1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:52.627 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.804+0000 7f91c1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51186/0 (socket says 172.21.15.57:51186) 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.804+0000 7f91c1e00640 1 -- 172.21.15.57:0/1230224006 learned_addr learned my addr 172.21.15.57:0/1230224006 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.805+0000 7f91c1e00640 1 -- 172.21.15.57:0/1230224006 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c407c9b0 con 0x7f91c407de00 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.807+0000 7f91c1e00640 1 --2- 172.21.15.57:0/1230224006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009820 tx=0x7f91b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2efc28faa15238 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.808+0000 7f91c1400640 1 -- 172.21.15.57:0/1230224006 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b4037070 con 0x7f91c407de00 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.808+0000 7f91c1400640 1 -- 172.21.15.57:0/1230224006 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91b4032900 con 0x7f91c407de00 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.808+0000 7f91c1400640 1 -- 172.21.15.57:0/1230224006 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b4030850 con 0x7f91c407de00 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 -- 172.21.15.57:0/1230224006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 msgr2=0x7f91c407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 --2- 172.21.15.57:0/1230224006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009820 tx=0x7f91b402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 -- 172.21.15.57:0/1230224006 shutdown_connections 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 --2- 172.21.15.57:0/1230224006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c407c260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 -- 172.21.15.57:0/1230224006 >> 172.21.15.57:0/1230224006 conn(0x7f91c4102e00 msgr2=0x7f91c4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 -- 172.21.15.57:0/1230224006 shutdown_connections 2024-09-12T02:54:52.628 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.809+0000 7f91c8c00640 1 -- 172.21.15.57:0/1230224006 wait complete. 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.810+0000 7f91c8c00640 1 Processor -- start 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.810+0000 7f91c8c00640 1 -- start start 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.810+0000 7f91c8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.810+0000 7f91c8c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f91c410dca0 con 0x7f91c407de00 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.810+0000 7f91c1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.811+0000 7f91c1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51190/0 (socket says 172.21.15.57:51190) 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.811+0000 7f91c1e00640 1 -- 172.21.15.57:0/741700553 learned_addr learned my addr 172.21.15.57:0/741700553 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.811+0000 7f91c1e00640 1 -- 172.21.15.57:0/741700553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c419dda0 con 0x7f91c407de00 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.811+0000 7f91c1e00640 1 --2- 172.21.15.57:0/741700553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f91b4032ce0 tx=0x7f91b4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b4042070 con 0x7f91c407de00 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91b40308c0 con 0x7f91c407de00 2024-09-12T02:54:52.629 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c419e030 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91b4037040 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c41a0d20 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91b4045070 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.812+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91b4047bd0 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.813+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c407c2e0 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.816+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f91b403c830 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.880+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f91c41a0ef0 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.882+0000 7f91baa00640 1 -- 172.21.15.57:0/741700553 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f91b403ca10 con 0x7f91c407de00 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.883+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 msgr2=0x7f91c419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.883+0000 7f91c8c00640 1 --2- 172.21.15.57:0/741700553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f91b4032ce0 tx=0x7f91b4032d10 comp rx=0 tx=0).stop 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.883+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 shutdown_connections 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.883+0000 7f91c8c00640 1 --2- 172.21.15.57:0/741700553 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f91c407de00 0x7f91c419d860 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:52.630 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.883+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 >> 172.21.15.57:0/741700553 conn(0x7f91c4102e00 msgr2=0x7f91c4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:52.631 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.884+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 shutdown_connections 2024-09-12T02:54:52.631 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:51.884+0000 7f91c8c00640 1 -- 172.21.15.57:0/741700553 wait complete. 2024-09-12T02:54:52.631 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T02:54:52.649 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:54:52.649 INFO:teuthology.orchestra.run.smithi057.stdout:Creating mgr... 2024-09-12T02:54:52.650 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T02:54:52.650 INFO:teuthology.orchestra.run.smithi057.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T02:54:53.252 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:52 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/741700553' entity='client.admin' 2024-09-12T02:54:53.298 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:54:53.298 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T02:54:53.537 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968.target.wants/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@.service. 2024-09-12T02:54:53.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:53 smithi057 systemd[1]: Starting Ceph mgr.x for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:54:54.155 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:53 smithi057 podman[30586]: 2024-09-12 02:54:53.784484742 +0000 UTC m=+0.026956513 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:54:54.155 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:53 smithi057 podman[30586]: 2024-09-12 02:54:53.893956517 +0000 UTC m=+0.136428286 container create c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T02:54:54.155 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 podman[30586]: 2024-09-12 02:54:54.079880389 +0000 UTC m=+0.322352159 container init c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T02:54:54.155 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 podman[30586]: 2024-09-12 02:54:54.08624385 +0000 UTC m=+0.328715620 container start c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:54:54.155 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 bash[30586]: c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 2024-09-12T02:54:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:54:54.188 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-09-12T02:54:54.204 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:54:54.204 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:54:54.204 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-09-12T02:54:54.204 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:54:54.225 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:54:54.225 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr to start... 2024-09-12T02:54:54.240 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr... 2024-09-12T02:54:54.505 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 systemd[1]: Started Ceph mgr.x for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:54:54.505 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:54.273+0000 7fb376a0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:54:54.505 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:54.345+0000 7fb376a0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:54:54.766 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:54.547+0000 7fb376a0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:54:54.766 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:54.632+0000 7fb376a0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:54:55.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3310982872' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:54:55.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:54.816+0000 7fb376a0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:54:55.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:55.006+0000 7fb376a0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:54:55.545 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:55.399+0000 7fb376a0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:54:55.545 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:55.473+0000 7fb376a0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:54:55.550 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:54:55.553 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:55.554 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:54:55.555 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:54:55.555 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:54:55.562 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:54:45:449413+0000", 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:54:55.563 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:54:55.564 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:54:45.450453+0000", 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.793+0000 7fcc68a00640 1 Processor -- start 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.793+0000 7fcc68a00640 1 -- start start 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.793+0000 7fcc68a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.793+0000 7fcc68a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410a250 con 0x7fcc64109880 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.794+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.794+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51226/0 (socket says 172.21.15.57:51226) 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.794+0000 7fcc61e00640 1 -- 172.21.15.57:0/3899026651 learned_addr learned my addr 172.21.15.57:0/3899026651 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:55.565 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.794+0000 7fcc61e00640 1 -- 172.21.15.57:0/3899026651 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6410aa80 con 0x7fcc64109880 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.795+0000 7fcc61e00640 1 --2- 172.21.15.57:0/3899026651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009820 tx=0x7fcc5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feea1247771fd1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.796+0000 7fcc60a00640 1 -- 172.21.15.57:0/3899026651 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc54037070 con 0x7fcc64109880 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.796+0000 7fcc60a00640 1 -- 172.21.15.57:0/3899026651 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc54032900 con 0x7fcc64109880 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.796+0000 7fcc60a00640 1 -- 172.21.15.57:0/3899026651 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc54030850 con 0x7fcc64109880 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.796+0000 7fcc68a00640 1 -- 172.21.15.57:0/3899026651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 msgr2=0x7fcc64109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.796+0000 7fcc68a00640 1 --2- 172.21.15.57:0/3899026651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009820 tx=0x7fcc5402a270 comp rx=0 tx=0).stop 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.797+0000 7fcc68a00640 1 -- 172.21.15.57:0/3899026651 shutdown_connections 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.797+0000 7fcc68a00640 1 --2- 172.21.15.57:0/3899026651 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc64109c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.797+0000 7fcc68a00640 1 -- 172.21.15.57:0/3899026651 >> 172.21.15.57:0/3899026651 conn(0x7fcc6407ce60 msgr2=0x7fcc6407d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.797+0000 7fcc68a00640 1 -- 172.21.15.57:0/3899026651 shutdown_connections 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.797+0000 7fcc68a00640 1 -- 172.21.15.57:0/3899026651 wait complete. 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc68a00640 1 Processor -- start 2024-09-12T02:54:55.566 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc68a00640 1 -- start start 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc68a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc68a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6410de30 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51228/0 (socket says 172.21.15.57:51228) 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.798+0000 7fcc61e00640 1 -- 172.21.15.57:0/3310982872 learned_addr learned my addr 172.21.15.57:0/3310982872 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.799+0000 7fcc61e00640 1 -- 172.21.15.57:0/3310982872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6419ddc0 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.799+0000 7fcc61e00640 1 --2- 172.21.15.57:0/3310982872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc540097f0 tx=0x7fcc54030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.799+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc54037040 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.799+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6419e050 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.800+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc54040070 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.800+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a0d40 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.800+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc54045590 con 0x7fcc64109880 2024-09-12T02:54:55.567 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.800+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc54045770 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.800+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc540449b0 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.801+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6410dfb0 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.803+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcc540443c0 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.870+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc641a0f10 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.870+0000 7fcc5aa00640 1 -- 172.21.15.57:0/3310982872 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fcc5403b430 con 0x7fcc64109880 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 msgr2=0x7fcc6419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 --2- 172.21.15.57:0/3310982872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc540097f0 tx=0x7fcc54030d80 comp rx=0 tx=0).stop 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 shutdown_connections 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 --2- 172.21.15.57:0/3310982872 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc64109880 0x7fcc6419d880 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 >> 172.21.15.57:0/3310982872 conn(0x7fcc6407ce60 msgr2=0x7fcc64106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.871+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 shutdown_connections 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:54.872+0000 7fcc68a00640 1 -- 172.21.15.57:0/3310982872 wait complete. 2024-09-12T02:54:55.568 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (1/15)... 2024-09-12T02:54:55.869 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:55.695+0000 7fb376a0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:54:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:56.837+0000 7fb376a0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:54:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:56.920+0000 7fb376a0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:54:58.006 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:57.900+0000 7fb376a0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:54:58.304 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2881250923' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:54:58.304 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.048+0000 7fb376a0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:54:58.305 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.229+0000 7fb376a0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:54:58.305 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.304+0000 7fb376a0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:54:58.571 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.486+0000 7fb376a0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:54:58.827 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.571+0000 7fb376a0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:54:58.827 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.645+0000 7fb376a0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.834 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.835 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:54:58.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:58.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:54:58.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:54:58.836 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:54:58.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:54:58.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:54:58.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:54:58.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:54:45:449413+0000", 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:54:58.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:54:45.450453+0000", 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:54:58.849 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.033+0000 7f0abda00640 1 Processor -- start 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.033+0000 7f0abda00640 1 -- start start 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.033+0000 7f0abda00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.033+0000 7f0abda00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab8107e70 con 0x7f0ab81074a0 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.034+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.034+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40816/0 (socket says 172.21.15.57:40816) 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.034+0000 7f0ab7e00640 1 -- 172.21.15.57:0/1684026547 learned_addr learned my addr 172.21.15.57:0/1684026547 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.034+0000 7f0ab7e00640 1 -- 172.21.15.57:0/1684026547 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab81086a0 con 0x7f0ab81074a0 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.035+0000 7f0ab7e00640 1 --2- 172.21.15.57:0/1684026547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009820 tx=0x7f0aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb32b6f07049b669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:58.850 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.036+0000 7f0ab7400640 1 -- 172.21.15.57:0/1684026547 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8037070 con 0x7f0ab81074a0 2024-09-12T02:54:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.036+0000 7f0ab7400640 1 -- 172.21.15.57:0/1684026547 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0aa8032900 con 0x7f0ab81074a0 2024-09-12T02:54:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.036+0000 7f0ab7400640 1 -- 172.21.15.57:0/1684026547 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8030850 con 0x7f0ab81074a0 2024-09-12T02:54:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.036+0000 7f0abda00640 1 -- 172.21.15.57:0/1684026547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 msgr2=0x7f0ab81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.036+0000 7f0abda00640 1 --2- 172.21.15.57:0/1684026547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009820 tx=0x7f0aa802a270 comp rx=0 tx=0).stop 2024-09-12T02:54:58.852 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.037+0000 7f0abda00640 1 -- 172.21.15.57:0/1684026547 shutdown_connections 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.037+0000 7f0abda00640 1 --2- 172.21.15.57:0/1684026547 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab81078a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.037+0000 7f0abda00640 1 -- 172.21.15.57:0/1684026547 >> 172.21.15.57:0/1684026547 conn(0x7f0ab8102c90 msgr2=0x7f0ab81050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.037+0000 7f0abda00640 1 -- 172.21.15.57:0/1684026547 shutdown_connections 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.037+0000 7f0abda00640 1 -- 172.21.15.57:0/1684026547 wait complete. 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0abda00640 1 Processor -- start 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0abda00640 1 -- start start 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0abda00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0abda00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab810ba50 con 0x7f0ab81074a0 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0ab7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40832/0 (socket says 172.21.15.57:40832) 2024-09-12T02:54:58.853 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.038+0000 7f0ab7e00640 1 -- 172.21.15.57:0/2881250923 learned_addr learned my addr 172.21.15.57:0/2881250923 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0ab7e00640 1 -- 172.21.15.57:0/2881250923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab819b9b0 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0ab7e00640 1 --2- 172.21.15.57:0/2881250923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097f0 tx=0x7f0aa8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa8037040 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0aa8040070 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aa803a390 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.039+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab819bc40 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.040+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab819e930 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.040+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0aa803a5e0 con 0x7f0ab81074a0 2024-09-12T02:54:58.854 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.040+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0aa8045950 con 0x7f0ab81074a0 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.041+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab810bbd0 con 0x7f0ab81074a0 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.045+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0aa8045460 con 0x7f0ab81074a0 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.110+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0ab819eb00 con 0x7f0ab81074a0 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.110+0000 7f0ab5600640 1 -- 172.21.15.57:0/2881250923 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f0aa803b430 con 0x7f0ab81074a0 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 msgr2=0x7f0ab819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 --2- 172.21.15.57:0/2881250923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097f0 tx=0x7f0aa8030d80 comp rx=0 tx=0).stop 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 shutdown_connections 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 --2- 172.21.15.57:0/2881250923 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0ab81074a0 0x7f0ab819b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 >> 172.21.15.57:0/2881250923 conn(0x7f0ab8102c90 msgr2=0x7f0ab8103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 shutdown_connections 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:54:58.112+0000 7f0abda00640 1 -- 172.21.15.57:0/2881250923 wait complete. 2024-09-12T02:54:58.855 INFO:teuthology.orchestra.run.smithi057.stdout:mgr not available, waiting (2/15)... 2024-09-12T02:54:59.098 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.835+0000 7fb376a0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:54:59.098 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.916+0000 7fb376a0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:54:59.098 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:58.998+0000 7fb376a0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:54:59.098 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:59 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:59.098+0000 7fb376a0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:54:59.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:59 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:59.171+0000 7fb376a0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:54:59.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:54:59 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:54:59.243+0000 7fb376a0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:00.191+0000 7fb376a0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: Activating manager daemon x 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: mgrmap e2: x(active, starting, since 0.00454408s) 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:55:00.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: Manager daemon x is now available 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' 2024-09-12T02:55:00.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:00 smithi057 ceph-mon[30361]: from='mgr.14100 172.21.15.57:0/2170099715' entity='mgr.x' 2024-09-12T02:55:02.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:02 smithi057 ceph-mon[30361]: mgrmap e3: x(active, since 1.00925s) 2024-09-12T02:55:02.283 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:02 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3522770601' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:55:02.291 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.292 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:55:02.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:55:02.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:55:02.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:55:02.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:55:02.293 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:55:02.301 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:55:02.301 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:55:02.301 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.301 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.302 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:54:45:449413+0000", 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:55:02.303 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:54:45.450453+0000", 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.410+0000 7f1dbb000640 1 Processor -- start 2024-09-12T02:55:02.304 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.410+0000 7f1dbb000640 1 -- start start 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1dbb000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1dbb000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1db41080f0 con 0x7f1db4107720 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1db3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1db3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40888/0 (socket says 172.21.15.57:40888) 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1db3e00640 1 -- 172.21.15.57:0/1474768965 learned_addr learned my addr 172.21.15.57:0/1474768965 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.411+0000 7f1db3e00640 1 -- 172.21.15.57:0/1474768965 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db41088d0 con 0x7f1db4107720 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.412+0000 7f1db3e00640 1 --2- 172.21.15.57:0/1474768965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009820 tx=0x7f1da402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec704da0d0339834 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1db2a00640 1 -- 172.21.15.57:0/1474768965 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da4037070 con 0x7f1db4107720 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1db2a00640 1 -- 172.21.15.57:0/1474768965 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1da4032900 con 0x7f1db4107720 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1db2a00640 1 -- 172.21.15.57:0/1474768965 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da4030850 con 0x7f1db4107720 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1dbb000640 1 -- 172.21.15.57:0/1474768965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 msgr2=0x7f1db4107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1dbb000640 1 --2- 172.21.15.57:0/1474768965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009820 tx=0x7f1da402a270 comp rx=0 tx=0).stop 2024-09-12T02:55:02.305 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1dbb000640 1 -- 172.21.15.57:0/1474768965 shutdown_connections 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1dbb000640 1 --2- 172.21.15.57:0/1474768965 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db4107b20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.413+0000 7f1dbb000640 1 -- 172.21.15.57:0/1474768965 >> 172.21.15.57:0/1474768965 conn(0x7f1db4102ef0 msgr2=0x7f1db4105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.414+0000 7f1dbb000640 1 -- 172.21.15.57:0/1474768965 shutdown_connections 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.414+0000 7f1dbb000640 1 -- 172.21.15.57:0/1474768965 wait complete. 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.414+0000 7f1dbb000640 1 Processor -- start 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1dbb000640 1 -- start start 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1dbb000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1dbb000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410bc80 con 0x7f1db4107720 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1db3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1db3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40890/0 (socket says 172.21.15.57:40890) 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.415+0000 7f1db3e00640 1 -- 172.21.15.57:0/3522770601 learned_addr learned my addr 172.21.15.57:0/3522770601 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:02.306 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1db3e00640 1 -- 172.21.15.57:0/3522770601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db407e8c0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1db3e00640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009940 tx=0x7f1da4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da4042070 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1da40308c0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db407cab0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1da4037040 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.416+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db407cff0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.417+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f1da4045070 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.417+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db4107ba0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.418+0000 7f1db0c00640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 0x7f1d9803b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.418+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1da406bda0 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.418+0000 7f1db3400640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 0x7f1d9803b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.419+0000 7f1db3400640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 0x7f1d9803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.424+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1da403cd90 con 0x7f1db4107720 2024-09-12T02:55:02.307 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.676+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1db407d830 con 0x7f1db4107720 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.677+0000 7f1db0c00640 1 -- 172.21.15.57:0/3522770601 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1da402fd90 con 0x7f1db4107720 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 msgr2=0x7f1d9803b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 0x7f1d9803b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1da00098c0 tx=0x7f1da0006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 msgr2=0x7f1db407e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009940 tx=0x7f1da4032ce0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 shutdown_connections 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f1d98038e10 0x7f1d9803b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 --2- 172.21.15.57:0/3522770601 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db4107720 0x7f1db407e380 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 >> 172.21.15.57:0/3522770601 conn(0x7f1db4102ef0 msgr2=0x7f1db4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 shutdown_connections 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:01.679+0000 7f1dbb000640 1 -- 172.21.15.57:0/3522770601 wait complete. 2024-09-12T02:55:02.308 INFO:teuthology.orchestra.run.smithi057.stdout:mgr is available 2024-09-12T02:55:03.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:03 smithi057 ceph-mon[30361]: mgrmap e4: x(active, since 2s) 2024-09-12T02:55:03.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:03 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4067467720' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout fsid = 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.57:3300,v1:172.21.15.57:6789] 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:55:03.618 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.776+0000 7f6fdbe00640 1 Processor -- start 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdbe00640 1 -- start start 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:03.619 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdbe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10a070 con 0x7f6fdc1096a0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40892/0 (socket says 172.21.15.57:40892) 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.777+0000 7f6fdaa00640 1 -- 172.21.15.57:0/188405210 learned_addr learned my addr 172.21.15.57:0/188405210 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.778+0000 7f6fdaa00640 1 -- 172.21.15.57:0/188405210 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc10a8a0 con 0x7f6fdc1096a0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.778+0000 7f6fdaa00640 1 --2- 172.21.15.57:0/188405210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009820 tx=0x7f6fcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c075fdff03183e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.779+0000 7f6fda000640 1 -- 172.21.15.57:0/188405210 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc037070 con 0x7f6fdc1096a0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.779+0000 7f6fda000640 1 -- 172.21.15.57:0/188405210 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fcc032900 con 0x7f6fdc1096a0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.779+0000 7f6fda000640 1 -- 172.21.15.57:0/188405210 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc030850 con 0x7f6fdc1096a0 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.779+0000 7f6fdbe00640 1 -- 172.21.15.57:0/188405210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 msgr2=0x7f6fdc109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.779+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/188405210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009820 tx=0x7f6fcc02a270 comp rx=0 tx=0).stop 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.780+0000 7f6fdbe00640 1 -- 172.21.15.57:0/188405210 shutdown_connections 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.780+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/188405210 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc109aa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:03.620 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.780+0000 7f6fdbe00640 1 -- 172.21.15.57:0/188405210 >> 172.21.15.57:0/188405210 conn(0x7f6fdc07cd10 msgr2=0x7f6fdc07d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.780+0000 7f6fdbe00640 1 -- 172.21.15.57:0/188405210 shutdown_connections 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.780+0000 7f6fdbe00640 1 -- 172.21.15.57:0/188405210 wait complete. 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdbe00640 1 Processor -- start 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdbe00640 1 -- start start 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdbe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc10dc50 con 0x7f6fdc1096a0 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40906/0 (socket says 172.21.15.57:40906) 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.781+0000 7f6fdaa00640 1 -- 172.21.15.57:0/4067467720 learned_addr learned my addr 172.21.15.57:0/4067467720 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fdaa00640 1 -- 172.21.15.57:0/4067467720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc19dbe0 con 0x7f6fdc1096a0 2024-09-12T02:55:03.621 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fdaa00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0097f0 tx=0x7f6fcc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc037040 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fcc040070 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc03a390 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc19de70 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.782+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a0b60 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.784+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6fcc03b430 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.784+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc10ddd0 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.784+0000 7f6fd3e00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 0x7f6fc003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.784+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fcc06c080 con 0x7f6fdc1096a0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.784+0000 7f6fd2000640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 0x7f6fc003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.785+0000 7f6fd2000640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 0x7f6fc003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40098c0 tx=0x7f6fc4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:03.622 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.789+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fcc06bde0 con 0x7f6fdc1096a0 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.977+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f6fdc109b20 con 0x7f6fdc1096a0 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.978+0000 7f6fd3e00640 1 -- 172.21.15.57:0/4067467720 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f6fcc032900 con 0x7f6fdc1096a0 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 msgr2=0x7f6fc003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 0x7f6fc003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40098c0 tx=0x7f6fc4006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 msgr2=0x7f6fdc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0097f0 tx=0x7f6fcc030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 shutdown_connections 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6fc0038f70 0x7f6fc003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 --2- 172.21.15.57:0/4067467720 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6fdc1096a0 0x7f6fdc19d6a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 >> 172.21.15.57:0/4067467720 conn(0x7f6fdc07cd10 msgr2=0x7f6fdc10bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 shutdown_connections 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:02.980+0000 7f6fdbe00640 1 -- 172.21.15.57:0/4067467720 wait complete. 2024-09-12T02:55:03.623 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling cephadm module... 2024-09-12T02:55:04.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1372350632' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T02:55:04.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-12T02:55:04.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-12T02:55:04.885 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:04.622+0000 7fe17ee10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:55:04.885 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:04.694+0000 7fe17ee10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:04.885 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:04.884+0000 7fe17ee10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:55:05.102 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.092+0000 7f5e73200640 1 Processor -- start 2024-09-12T02:55:05.102 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e73200640 1 -- start start 2024-09-12T02:55:05.102 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e73200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:05.102 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e73200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74107e60 con 0x7f5e74107490 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e71e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e71e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40912/0 (socket says 172.21.15.57:40912) 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.093+0000 7f5e71e00640 1 -- 172.21.15.57:0/4009768552 learned_addr learned my addr 172.21.15.57:0/4009768552 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.094+0000 7f5e71e00640 1 -- 172.21.15.57:0/4009768552 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e74108690 con 0x7f5e74107490 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e71e00640 1 --2- 172.21.15.57:0/4009768552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e80 tx=0x7f5e6402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f505a670b7d1bd22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e71400640 1 -- 172.21.15.57:0/4009768552 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e64036070 con 0x7f5e74107490 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e71400640 1 -- 172.21.15.57:0/4009768552 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e640318c0 con 0x7f5e74107490 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e71400640 1 -- 172.21.15.57:0/4009768552 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e64007ab0 con 0x7f5e74107490 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e73200640 1 -- 172.21.15.57:0/4009768552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 msgr2=0x7f5e74107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.095+0000 7f5e73200640 1 --2- 172.21.15.57:0/4009768552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009e80 tx=0x7f5e6402a7a0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 -- 172.21.15.57:0/4009768552 shutdown_connections 2024-09-12T02:55:05.103 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 --2- 172.21.15.57:0/4009768552 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e74107890 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 -- 172.21.15.57:0/4009768552 >> 172.21.15.57:0/4009768552 conn(0x7f5e74102c60 msgr2=0x7f5e74105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 -- 172.21.15.57:0/4009768552 shutdown_connections 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 -- 172.21.15.57:0/4009768552 wait complete. 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.096+0000 7f5e73200640 1 Processor -- start 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e73200640 1 -- start start 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e73200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e73200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7410ba40 con 0x7f5e74107490 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e71e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e71e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40926/0 (socket says 172.21.15.57:40926) 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.097+0000 7f5e71e00640 1 -- 172.21.15.57:0/1372350632 learned_addr learned my addr 172.21.15.57:0/1372350632 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e71e00640 1 -- 172.21.15.57:0/1372350632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7419b9d0 con 0x7f5e74107490 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e71e00640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5e6402acc0 tx=0x7f5e64031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:05.104 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e64036050 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e64037040 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e6403d8c0 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7419bc60 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.098+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7419e950 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.099+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5e64044020 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.099+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7410bbc0 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.100+0000 7f5e6b400640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 0x7f5e5803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.100+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e64042020 con 0x7f5e74107490 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.100+0000 7f5e69400640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 0x7f5e5803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.101+0000 7f5e69400640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 0x7f5e5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0098c0 tx=0x7f5e5c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:05.105 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.104+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e6404b020 con 0x7f5e74107490 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.338+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f5e7419efe0 con 0x7f5e74107490 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.391+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f5e6403bb30 con 0x7f5e74107490 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.392+0000 7f5e6b400640 1 -- 172.21.15.57:0/1372350632 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f5e6403b610 con 0x7f5e74107490 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 msgr2=0x7f5e5803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 0x7f5e5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0098c0 tx=0x7f5e5c006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 msgr2=0x7f5e7419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5e6402acc0 tx=0x7f5e64031ca0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 shutdown_connections 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f5e58038f70 0x7f5e5803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 --2- 172.21.15.57:0/1372350632 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e74107490 0x7f5e7419b490 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:05.106 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 >> 172.21.15.57:0/1372350632 conn(0x7f5e74102c60 msgr2=0x7f5e74103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:05.107 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 shutdown_connections 2024-09-12T02:55:05.107 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:04.394+0000 7f5e73200640 1 -- 172.21.15.57:0/1372350632 wait complete. 2024-09-12T02:55:05.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:04.967+0000 7fe17ee10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:55:05.147 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:05.146+0000 7fe17ee10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:05.402 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:05.317+0000 7fe17ee10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:55:05.707 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1372350632' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T02:55:05.707 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-mon[30361]: mgrmap e5: x(active, since 4s) 2024-09-12T02:55:06.005 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:05.706+0000 7fe17ee10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:55:06.005 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:05.781+0000 7fe17ee10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:55:06.005 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:06.004+0000 7fe17ee10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:55:06.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:06 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3189308329' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:55:06.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:55:06.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T02:55:06.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:55:06.532 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.601+0000 7fb56f200640 1 Processor -- start 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.601+0000 7fb56f200640 1 -- start start 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.601+0000 7fb56f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.601+0000 7fb56f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb57007c890 con 0x7fb57007de60 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.601+0000 7fb56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.602+0000 7fb56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40954/0 (socket says 172.21.15.57:40954) 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.602+0000 7fb56de00640 1 -- 172.21.15.57:0/3763947070 learned_addr learned my addr 172.21.15.57:0/3763947070 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.602+0000 7fb56de00640 1 -- 172.21.15.57:0/3763947070 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57007ca10 con 0x7fb57007de60 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.603+0000 7fb56de00640 1 --2- 172.21.15.57:0/3763947070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb560009820 tx=0x7fb56002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a9d35c6c41e0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.603+0000 7fb56d400640 1 -- 172.21.15.57:0/3763947070 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb560037070 con 0x7fb57007de60 2024-09-12T02:55:06.533 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.603+0000 7fb56d400640 1 -- 172.21.15.57:0/3763947070 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb560032900 con 0x7fb57007de60 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.603+0000 7fb56d400640 1 -- 172.21.15.57:0/3763947070 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb560030850 con 0x7fb57007de60 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 -- 172.21.15.57:0/3763947070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 msgr2=0x7fb57007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 --2- 172.21.15.57:0/3763947070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb560009820 tx=0x7fb56002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 -- 172.21.15.57:0/3763947070 shutdown_connections 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 --2- 172.21.15.57:0/3763947070 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb57007c2c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 -- 172.21.15.57:0/3763947070 >> 172.21.15.57:0/3763947070 conn(0x7fb570102c90 msgr2=0x7fb5701050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 -- 172.21.15.57:0/3763947070 shutdown_connections 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.604+0000 7fb56f200640 1 -- 172.21.15.57:0/3763947070 wait complete. 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.605+0000 7fb56f200640 1 Processor -- start 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.605+0000 7fb56f200640 1 -- start start 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:06.534 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701096c0 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40962/0 (socket says 172.21.15.57:40962) 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56de00640 1 -- 172.21.15.57:0/3189308329 learned_addr learned my addr 172.21.15.57:0/3189308329 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.606+0000 7fb56de00640 1 -- 172.21.15.57:0/3189308329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5701a1fe0 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb56de00640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb560009940 tx=0x7fb560032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb560042070 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb5600308c0 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5701a2270 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb560037040 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.607+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a4f60 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.608+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb560045070 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.608+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57007c340 con 0x7fb57007de60 2024-09-12T02:55:06.535 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.608+0000 7fb567400640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 0x7fb55403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.609+0000 7fb565400640 1 -- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 msgr2=0x7fb55403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.609+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb56006bfb0 con 0x7fb57007de60 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.609+0000 7fb565400640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 0x7fb55403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.614+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb56003cd90 con 0x7fb57007de60 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.810+0000 7fb565400640 1 -- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 msgr2=0x7fb55403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.810+0000 7fb565400640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 0x7fb55403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.855+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb57019af10 con 0x7fb57007de60 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.856+0000 7fb567400640 1 -- 172.21.15.57:0/3189308329 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb56003c820 con 0x7fb57007de60 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 msgr2=0x7fb55403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 0x7fb55403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 msgr2=0x7fb5701a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:06.536 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb560009940 tx=0x7fb560032ce0 comp rx=0 tx=0).stop 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 shutdown_connections 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7fb554038f70 0x7fb55403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 --2- 172.21.15.57:0/3189308329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb57007de60 0x7fb5701a1aa0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 >> 172.21.15.57:0/3189308329 conn(0x7fb570102c90 msgr2=0x7fb570105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 shutdown_connections 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:05.858+0000 7fb56f200640 1 -- 172.21.15.57:0/3189308329 wait complete. 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-09-12T02:55:06.537 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 5... 2024-09-12T02:55:07.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:07.157+0000 7fe17ee10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:55:07.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:07.240+0000 7fe17ee10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:55:08.506 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.192+0000 7fe17ee10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:55:08.506 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.334+0000 7fe17ee10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:55:08.506 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.505+0000 7fe17ee10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:08.760 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.580+0000 7fe17ee10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:55:08.760 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.759+0000 7fe17ee10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:55:09.106 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.843+0000 7fe17ee10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:55:09.106 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:08.917+0000 7fe17ee10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:55:09.106 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.105+0000 7fe17ee10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:55:09.368 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.185+0000 7fe17ee10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:55:09.368 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.267+0000 7fe17ee10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:55:09.368 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.367+0000 7fe17ee10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:09.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.440+0000 7fe17ee10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:55:09.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:09.513+0000 7fe17ee10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:10.463+0000 7fe17ee10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: Active manager daemon x restarted 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: Activating manager daemon x 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: mgrmap e6: x(active, starting, since 0.00447021s) 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:55:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: Manager daemon x is now available 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: Found migration_current of "None". Setting to last migration. 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:10 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:11.816 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:11 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:55:11.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:11 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:55:11.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:11 smithi057 ceph-mon[30361]: mgrmap e7: x(active, since 1.00699s) 2024-09-12T02:55:11.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:11 smithi057 ceph-mon[30361]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:55:11.817 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:11 smithi057 ceph-mon[30361]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.024+0000 7f6e7b000640 1 Processor -- start 2024-09-12T02:55:12.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.024+0000 7f6e7b000640 1 -- start start 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.025+0000 7f6e7b000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.025+0000 7f6e7b000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7410a2b0 con 0x7f6e741098e0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.025+0000 7f6e73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.025+0000 7f6e73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40968/0 (socket says 172.21.15.57:40968) 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.025+0000 7f6e73e00640 1 -- 172.21.15.57:0/464889217 learned_addr learned my addr 172.21.15.57:0/464889217 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.026+0000 7f6e73e00640 1 -- 172.21.15.57:0/464889217 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7410aa90 con 0x7f6e741098e0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.027+0000 7f6e73e00640 1 --2- 172.21.15.57:0/464889217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009820 tx=0x7f6e6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec900e6a1918d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e72a00640 1 -- 172.21.15.57:0/464889217 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e64037070 con 0x7f6e741098e0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e72a00640 1 -- 172.21.15.57:0/464889217 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e64032900 con 0x7f6e741098e0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e72a00640 1 -- 172.21.15.57:0/464889217 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e64030850 con 0x7f6e741098e0 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e7b000640 1 -- 172.21.15.57:0/464889217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 msgr2=0x7f6e74109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:12.592 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e7b000640 1 --2- 172.21.15.57:0/464889217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009820 tx=0x7f6e6402a270 comp rx=0 tx=0).stop 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e7b000640 1 -- 172.21.15.57:0/464889217 shutdown_connections 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e7b000640 1 --2- 172.21.15.57:0/464889217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e74109ce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.028+0000 7f6e7b000640 1 -- 172.21.15.57:0/464889217 >> 172.21.15.57:0/464889217 conn(0x7f6e7407cec0 msgr2=0x7f6e7407d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.029+0000 7f6e7b000640 1 -- 172.21.15.57:0/464889217 shutdown_connections 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.029+0000 7f6e7b000640 1 -- 172.21.15.57:0/464889217 wait complete. 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e7b000640 1 Processor -- start 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e7b000640 1 -- start start 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e7b000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e7b000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7410de40 con 0x7f6e741098e0 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e73e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40974/0 (socket says 172.21.15.57:40974) 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.030+0000 7f6e73e00640 1 -- 172.21.15.57:0/3856523766 learned_addr learned my addr 172.21.15.57:0/3856523766 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:12.593 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e73e00640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7419ddd0 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e73e00640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e640097f0 tx=0x7f6e64030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e64037040 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e64040070 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6403a390 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.031+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7419e060 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.032+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e741a0d50 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.033+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6e6403b430 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.034+0000 7f6e70c00640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.034+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6e5803bb90 con 0x7f6e58038fc0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.034+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e6406c0d0 con 0x7f6e741098e0 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.034+0000 7f6e73400640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.034+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:55:12.594 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.235+0000 7f6e73400640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.235+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.636+0000 7f6e73400640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:07.636+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:08.437+0000 7f6e73400640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:08.437+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:10.038+0000 7f6e73400640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/1326814975 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:10.039+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:10.468+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f6e64032900 con 0x7f6e741098e0 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:10.468+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 msgr2=0x7f6e5803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:10.468+0000 7f6e70c00640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.470+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f6e6406cf70 con 0x7f6e741098e0 2024-09-12T02:55:12.595 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.470+0000 7f6e70c00640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 0x7f6e5803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.470+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6e64043a70 con 0x7f6e5803c8f0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.471+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 0x7f6e5803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.472+0000 7f6e73400640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 0x7f6e5803ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e60003980 tx=0x7f6e600092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.473+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6e64043a70 con 0x7f6e5803c8f0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.480+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6e7410dfc0 con 0x7f6e5803c8f0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.480+0000 7f6e70c00640 1 -- 172.21.15.57:0/3856523766 <== mgr.14118 v2:172.21.15.57:6800/522425633 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f6e7410dfc0 con 0x7f6e5803c8f0 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 msgr2=0x7f6e5803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 0x7f6e5803ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6e60003980 tx=0x7f6e600092b0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 msgr2=0x7f6e7419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e640097f0 tx=0x7f6e64030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 shutdown_connections 2024-09-12T02:55:12.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f6e5803c8f0 0x7f6e5803ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:6800/1326814975,v1:172.21.15.57:6801/1326814975] conn(0x7f6e58038fc0 0x7f6e5803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 --2- 172.21.15.57:0/3856523766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e741098e0 0x7f6e7419d890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 >> 172.21.15.57:0/3856523766 conn(0x7f6e7407cec0 msgr2=0x7f6e74106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 shutdown_connections 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:11.481+0000 7f6e7b000640 1 -- 172.21.15.57:0/3856523766 wait complete. 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 5 is available 2024-09-12T02:55:12.597 INFO:teuthology.orchestra.run.smithi057.stdout:Setting orchestrator backend to cephadm... 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: mgrmap e8: x(active, since 2s) 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:13 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:14.022 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.099+0000 7f380b000640 1 Processor -- start 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.099+0000 7f380b000640 1 -- start start 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f380b000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f380b000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f38041080b0 con 0x7f38041076e0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f3803e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f3803e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35014/0 (socket says 172.21.15.57:35014) 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f3803e00640 1 -- 172.21.15.57:0/1153662954 learned_addr learned my addr 172.21.15.57:0/1153662954 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.100+0000 7f3803e00640 1 -- 172.21.15.57:0/1153662954 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3804108890 con 0x7f38041076e0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.101+0000 7f3803e00640 1 --2- 172.21.15.57:0/1153662954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009820 tx=0x7f37f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7e02266d86b07b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f3802a00640 1 -- 172.21.15.57:0/1153662954 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f4037070 con 0x7f38041076e0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f3802a00640 1 -- 172.21.15.57:0/1153662954 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f37f4032900 con 0x7f38041076e0 2024-09-12T02:55:14.023 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f3802a00640 1 -- 172.21.15.57:0/1153662954 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f4030850 con 0x7f38041076e0 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f380b000640 1 -- 172.21.15.57:0/1153662954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 msgr2=0x7f3804107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f380b000640 1 --2- 172.21.15.57:0/1153662954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009820 tx=0x7f37f402a270 comp rx=0 tx=0).stop 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f380b000640 1 -- 172.21.15.57:0/1153662954 shutdown_connections 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f380b000640 1 --2- 172.21.15.57:0/1153662954 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f3804107ae0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.102+0000 7f380b000640 1 -- 172.21.15.57:0/1153662954 >> 172.21.15.57:0/1153662954 conn(0x7f3804102e90 msgr2=0x7f38041052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.103+0000 7f380b000640 1 -- 172.21.15.57:0/1153662954 shutdown_connections 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.103+0000 7f380b000640 1 -- 172.21.15.57:0/1153662954 wait complete. 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f380b000640 1 Processor -- start 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f380b000640 1 -- start start 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f380b000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f380b000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f380410bc40 con 0x7f38041076e0 2024-09-12T02:55:14.024 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f3803e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f3803e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35022/0 (socket says 172.21.15.57:35022) 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.104+0000 7f3803e00640 1 -- 172.21.15.57:0/2756546932 learned_addr learned my addr 172.21.15.57:0/2756546932 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f3803e00640 1 -- 172.21.15.57:0/2756546932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380407e8c0 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f3803e00640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097f0 tx=0x7f37f4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f4037040 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380407cab0 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f37f4040070 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.105+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f403a390 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.106+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380407cf90 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.107+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f37f403b430 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.107+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3804107b60 con 0x7f38041076e0 2024-09-12T02:55:14.025 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.107+0000 7f3800c00640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 0x7f37e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.108+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f37f406c4a0 con 0x7f38041076e0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.108+0000 7f3803400640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 0x7f37e803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.108+0000 7f3803400640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 0x7f37e803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098c0 tx=0x7f37ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.113+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37f406be30 con 0x7f38041076e0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.316+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3804104120 con 0x7f37e8038fc0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.321+0000 7f3800c00640 1 -- 172.21.15.57:0/2756546932 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3804104120 con 0x7f37e8038fc0 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 msgr2=0x7f37e803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 0x7f37e803b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098c0 tx=0x7f37ec006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 msgr2=0x7f380407e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f40097f0 tx=0x7f37f4030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 shutdown_connections 2024-09-12T02:55:14.026 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f37e8038fc0 0x7f37e803b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:14.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 --2- 172.21.15.57:0/2756546932 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f38041076e0 0x7f380407e380 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:14.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 >> 172.21.15.57:0/2756546932 conn(0x7f3804102e90 msgr2=0x7f3804103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:14.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 shutdown_connections 2024-09-12T02:55:14.027 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:13.324+0000 7f380b000640 1 -- 172.21.15.57:0/2756546932 wait complete. 2024-09-12T02:55:15.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:15.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:15.013+0000 7fe0d7600640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:55:15] ENGINE Error in HTTPServer.serve 2024-09-12T02:55:15.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Traceback (most recent call last): 2024-09-12T02:55:15.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._run(expiration_interval) 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.do_handshake() 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:55:15.371 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-12T02:55:15.372 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:55:15.372 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.521+0000 7efd27000640 1 Processor -- start 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd27000640 1 -- start start 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd27000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd27000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010a250 con 0x7efd20109880 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35028/0 (socket says 172.21.15.57:35028) 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.522+0000 7efd1fe00640 1 -- 172.21.15.57:0/899343675 learned_addr learned my addr 172.21.15.57:0/899343675 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.523+0000 7efd1fe00640 1 -- 172.21.15.57:0/899343675 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2010aa80 con 0x7efd20109880 2024-09-12T02:55:15.411 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd1fe00640 1 --2- 172.21.15.57:0/899343675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd10009820 tx=0x7efd1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c111da81f5625b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd1ea00640 1 -- 172.21.15.57:0/899343675 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd10037070 con 0x7efd20109880 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd1ea00640 1 -- 172.21.15.57:0/899343675 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd10032900 con 0x7efd20109880 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd1ea00640 1 -- 172.21.15.57:0/899343675 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd10030850 con 0x7efd20109880 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd27000640 1 -- 172.21.15.57:0/899343675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 msgr2=0x7efd20109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd27000640 1 --2- 172.21.15.57:0/899343675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd10009820 tx=0x7efd1002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd27000640 1 -- 172.21.15.57:0/899343675 shutdown_connections 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd27000640 1 --2- 172.21.15.57:0/899343675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd20109c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.524+0000 7efd27000640 1 -- 172.21.15.57:0/899343675 >> 172.21.15.57:0/899343675 conn(0x7efd2007ce60 msgr2=0x7efd2007d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.525+0000 7efd27000640 1 -- 172.21.15.57:0/899343675 shutdown_connections 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.525+0000 7efd27000640 1 -- 172.21.15.57:0/899343675 wait complete. 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.525+0000 7efd27000640 1 Processor -- start 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd27000640 1 -- start start 2024-09-12T02:55:15.412 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd27000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd27000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010de30 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35034/0 (socket says 172.21.15.57:35034) 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd1fe00640 1 -- 172.21.15.57:0/3769083849 learned_addr learned my addr 172.21.15.57:0/3769083849 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.526+0000 7efd1fe00640 1 -- 172.21.15.57:0/3769083849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2019dde0 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd1fe00640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd100097f0 tx=0x7efd10030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd10037040 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd10040070 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2019e070 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd1003a390 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.527+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd201a0d60 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.528+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2010dfb0 con 0x7efd20109880 2024-09-12T02:55:15.413 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.528+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efd1003b430 con 0x7efd20109880 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.529+0000 7efd1cc00640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 0x7efd0403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.529+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efd1006c4e0 con 0x7efd20109880 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.529+0000 7efd1f400640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 0x7efd0403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.530+0000 7efd1f400640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 0x7efd0403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0098c0 tx=0x7efd0c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.534+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd1006be30 con 0x7efd20109880 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.719+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7efd201073e0 con 0x7efd04038fc0 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.720+0000 7efd1cc00640 1 -- 172.21.15.57:0/3769083849 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7efd201073e0 con 0x7efd04038fc0 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 msgr2=0x7efd0403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 0x7efd0403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0098c0 tx=0x7efd0c006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 msgr2=0x7efd2019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efd100097f0 tx=0x7efd10030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 shutdown_connections 2024-09-12T02:55:15.414 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7efd04038fc0 0x7efd0403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.415 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 --2- 172.21.15.57:0/3769083849 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efd20109880 0x7efd2019d8a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:15.415 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 >> 172.21.15.57:0/3769083849 conn(0x7efd2007ce60 msgr2=0x7efd20106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:15.415 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 shutdown_connections 2024-09-12T02:55:15.415 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:14.722+0000 7efd27000640 1 -- 172.21.15.57:0/3769083849 wait complete. 2024-09-12T02:55:15.415 INFO:teuthology.orchestra.run.smithi057.stdout:Generating ssh key... 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:14] ENGINE Bus STARTING 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:14] ENGINE Serving on http://172.21.15.57:8765 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:15] ENGINE Serving on https://172.21.15.57:7150 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:15] ENGINE Bus STARTED 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:15] ENGINE Error in HTTPServer.serve 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:55:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-09-12T02:55:16.371 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:55:16.757 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Generating public/private rsa key pair. 2024-09-12T02:55:16.757 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Your identification has been saved in /tmp/tmpjetm0itt/key 2024-09-12T02:55:16.757 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Your public key has been saved in /tmp/tmpjetm0itt/key.pub 2024-09-12T02:55:16.757 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: The key fingerprint is: 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: SHA256:7dOZiMohde6ouJjc0j4RKzEvSHsX5bDblmpcgeFMjxA ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: The key's randomart image is: 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: +---[RSA 3072]----+ 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: | E. | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: | . + . | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: | = X | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: | + . B + . | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: |o = o = S . | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: |.+ = + B o o o | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: | = = = o + + | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: |.+.+ * = . | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: |o.=++.+ . | 2024-09-12T02:55:16.758 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: +----[SHA256]-----+ 2024-09-12T02:55:17.093 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.910+0000 7fe9c6000640 1 Processor -- start 2024-09-12T02:55:17.093 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.910+0000 7fe9c6000640 1 -- start start 2024-09-12T02:55:17.093 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:17.093 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c6000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c010a070 con 0x7fe9c01096a0 2024-09-12T02:55:17.093 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35038/0 (socket says 172.21.15.57:35038) 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c4c00640 1 -- 172.21.15.57:0/3497325005 learned_addr learned my addr 172.21.15.57:0/3497325005 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.911+0000 7fe9c4c00640 1 -- 172.21.15.57:0/3497325005 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c010a8a0 con 0x7fe9c01096a0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.912+0000 7fe9c4c00640 1 --2- 172.21.15.57:0/3497325005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009820 tx=0x7fe9b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b598efdfc2c44f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9bfe00640 1 -- 172.21.15.57:0/3497325005 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0037070 con 0x7fe9c01096a0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9bfe00640 1 -- 172.21.15.57:0/3497325005 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9b0032900 con 0x7fe9c01096a0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9bfe00640 1 -- 172.21.15.57:0/3497325005 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0030850 con 0x7fe9c01096a0 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 -- 172.21.15.57:0/3497325005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 msgr2=0x7fe9c0109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 --2- 172.21.15.57:0/3497325005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009820 tx=0x7fe9b002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 -- 172.21.15.57:0/3497325005 shutdown_connections 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 --2- 172.21.15.57:0/3497325005 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c0109aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 -- 172.21.15.57:0/3497325005 >> 172.21.15.57:0/3497325005 conn(0x7fe9c007cd10 msgr2=0x7fe9c007d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:17.094 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.913+0000 7fe9c6000640 1 -- 172.21.15.57:0/3497325005 shutdown_connections 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.914+0000 7fe9c6000640 1 -- 172.21.15.57:0/3497325005 wait complete. 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.914+0000 7fe9c6000640 1 Processor -- start 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c6000640 1 -- start start 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c6000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c010dc50 con 0x7fe9c01096a0 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35040/0 (socket says 172.21.15.57:35040) 2024-09-12T02:55:17.095 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c4c00640 1 -- 172.21.15.57:0/4058081307 learned_addr learned my addr 172.21.15.57:0/4058081307 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.915+0000 7fe9c4c00640 1 -- 172.21.15.57:0/4058081307 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c019dbe0 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9c4c00640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009940 tx=0x7fe9b0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0042070 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9b00308c0 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c019de70 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9b0037040 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.916+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c01a0b60 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.917+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe9b0045070 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.917+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c010ddd0 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.918+0000 7fe9be000640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 0x7fe9a403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.918+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9b006c0b0 con 0x7fe9c01096a0 2024-09-12T02:55:17.096 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.918+0000 7fe9b7e00640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 0x7fe9a403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.919+0000 7fe9b7e00640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 0x7fe9a403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:15.923+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9b004b170 con 0x7fe9c01096a0 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.107+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe9c010c1d0 con 0x7fe9a4038fc0 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.432+0000 7fe9be000640 1 -- 172.21.15.57:0/4058081307 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe9c010c1d0 con 0x7fe9a4038fc0 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 msgr2=0x7fe9a403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 0x7fe9a403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a8006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 msgr2=0x7fe9c019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009940 tx=0x7fe9b0032ce0 comp rx=0 tx=0).stop 2024-09-12T02:55:17.097 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 shutdown_connections 2024-09-12T02:55:17.098 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fe9a4038fc0 0x7fe9a403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:17.098 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 --2- 172.21.15.57:0/4058081307 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9c01096a0 0x7fe9c019d6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:17.098 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 >> 172.21.15.57:0/4058081307 conn(0x7fe9c007cd10 msgr2=0x7fe9c010bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:17.098 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 shutdown_connections 2024-09-12T02:55:17.098 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:16.435+0000 7fe9c6000640 1 -- 172.21.15.57:0/4058081307 wait complete. 2024-09-12T02:55:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:17 smithi057 ceph-mon[30361]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:17 smithi057 ceph-mon[30361]: Generating ssh key... 2024-09-12T02:55:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:17 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:17 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.660+0000 7f5d6e600640 1 Processor -- start 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.660+0000 7f5d6e600640 1 -- start start 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.660+0000 7f5d6e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.660+0000 7f5d6e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810a250 con 0x7f5d68109880 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.661+0000 7f5d67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.661+0000 7f5d67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35052/0 (socket says 172.21.15.57:35052) 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.661+0000 7f5d67400640 1 -- 172.21.15.57:0/1032944062 learned_addr learned my addr 172.21.15.57:0/1032944062 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.661+0000 7f5d67400640 1 -- 172.21.15.57:0/1032944062 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6810aa80 con 0x7f5d68109880 2024-09-12T02:55:18.482 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.662+0000 7f5d67400640 1 --2- 172.21.15.57:0/1032944062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009820 tx=0x7f5d5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198f801bd20b839 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.662+0000 7f5d66000640 1 -- 172.21.15.57:0/1032944062 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d58037070 con 0x7f5d68109880 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.662+0000 7f5d66000640 1 -- 172.21.15.57:0/1032944062 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d58032900 con 0x7f5d68109880 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.662+0000 7f5d66000640 1 -- 172.21.15.57:0/1032944062 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d58030850 con 0x7f5d68109880 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 -- 172.21.15.57:0/1032944062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 msgr2=0x7f5d68109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 --2- 172.21.15.57:0/1032944062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009820 tx=0x7f5d5802a270 comp rx=0 tx=0).stop 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 -- 172.21.15.57:0/1032944062 shutdown_connections 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 --2- 172.21.15.57:0/1032944062 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d68109c80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 -- 172.21.15.57:0/1032944062 >> 172.21.15.57:0/1032944062 conn(0x7f5d6807ce60 msgr2=0x7f5d6807d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 -- 172.21.15.57:0/1032944062 shutdown_connections 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.663+0000 7f5d6e600640 1 -- 172.21.15.57:0/1032944062 wait complete. 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d6e600640 1 Processor -- start 2024-09-12T02:55:18.483 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d6e600640 1 -- start start 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d6e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d6e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6810de30 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35068/0 (socket says 172.21.15.57:35068) 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.664+0000 7f5d67400640 1 -- 172.21.15.57:0/2193334692 learned_addr learned my addr 172.21.15.57:0/2193334692 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d67400640 1 -- 172.21.15.57:0/2193334692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6819ddc0 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d67400640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097f0 tx=0x7f5d58030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d58037040 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d58040070 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d5803a390 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.665+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6819e050 con 0x7f5d68109880 2024-09-12T02:55:18.484 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.666+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d681a0d40 con 0x7f5d68109880 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.667+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5d5803b430 con 0x7f5d68109880 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.667+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d68109d00 con 0x7f5d68109880 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.667+0000 7f5d5fe00640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 0x7f5d4c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.667+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5d5806c4e0 con 0x7f5d68109880 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.667+0000 7f5d66a00640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 0x7f5d4c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.668+0000 7f5d66a00640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 0x7f5d4c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d540098c0 tx=0x7f5d54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.672+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d5806be30 con 0x7f5d68109880 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.860+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f5d681073e0 con 0x7f5d4c038fc0 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.860+0000 7f5d5fe00640 1 -- 172.21.15.57:0/2193334692 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f5d681073e0 con 0x7f5d4c038fc0 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 msgr2=0x7f5d4c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 0x7f5d4c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d540098c0 tx=0x7f5d54006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 msgr2=0x7f5d6819d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097f0 tx=0x7f5d58030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 shutdown_connections 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f5d4c038fc0 0x7f5d4c03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 --2- 172.21.15.57:0/2193334692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d68109880 0x7f5d6819d880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 >> 172.21.15.57:0/2193334692 conn(0x7f5d6807ce60 msgr2=0x7f5d68106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 shutdown_connections 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:17.863+0000 7f5d6e600640 1 -- 172.21.15.57:0/2193334692 wait complete. 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:55:18.486 INFO:teuthology.orchestra.run.smithi057.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T02:55:18.487 INFO:teuthology.orchestra.run.smithi057.stdout:Adding host smithi057... 2024-09-12T02:55:18.758 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:18 smithi057 ceph-mon[30361]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:19 smithi057 ceph-mon[30361]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:21.732 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:21 smithi057 ceph-mon[30361]: Deploying cephadm binary to smithi057 2024-09-12T02:55:24.449 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Added host 'smithi057' with addr '172.21.15.57' 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f456d000640 1 Processor -- start 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f456d000640 1 -- start start 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f456d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f456d000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f456810a070 con 0x7f45681096a0 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f4567400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f4567400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41950/0 (socket says 172.21.15.57:41950) 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.976+0000 7f4567400640 1 -- 172.21.15.57:0/533189454 learned_addr learned my addr 172.21.15.57:0/533189454 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:24.450 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.977+0000 7f4567400640 1 -- 172.21.15.57:0/533189454 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456810a8a0 con 0x7f45681096a0 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f4567400640 1 --2- 172.21.15.57:0/533189454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4558009820 tx=0x7f455802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6abed26c8104ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f4566a00640 1 -- 172.21.15.57:0/533189454 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558037070 con 0x7f45681096a0 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f4566a00640 1 -- 172.21.15.57:0/533189454 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4558032900 con 0x7f45681096a0 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f4566a00640 1 -- 172.21.15.57:0/533189454 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558030850 con 0x7f45681096a0 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f456d000640 1 -- 172.21.15.57:0/533189454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 msgr2=0x7f4568109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.978+0000 7f456d000640 1 --2- 172.21.15.57:0/533189454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4558009820 tx=0x7f455802a270 comp rx=0 tx=0).stop 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.979+0000 7f456d000640 1 -- 172.21.15.57:0/533189454 shutdown_connections 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.979+0000 7f456d000640 1 --2- 172.21.15.57:0/533189454 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f4568109aa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.979+0000 7f456d000640 1 -- 172.21.15.57:0/533189454 >> 172.21.15.57:0/533189454 conn(0x7f456807cd10 msgr2=0x7f456807d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.979+0000 7f456d000640 1 -- 172.21.15.57:0/533189454 shutdown_connections 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.979+0000 7f456d000640 1 -- 172.21.15.57:0/533189454 wait complete. 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f456d000640 1 Processor -- start 2024-09-12T02:55:24.451 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f456d000640 1 -- start start 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f456d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f456d000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f456810dc50 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f4567400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f4567400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41966/0 (socket says 172.21.15.57:41966) 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.980+0000 7f4567400640 1 -- 172.21.15.57:0/2963991181 learned_addr learned my addr 172.21.15.57:0/2963991181 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f4567400640 1 -- 172.21.15.57:0/2963991181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456819dbe0 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f4567400640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4558009940 tx=0x7f4558032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558042070 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f45580308c0 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4558037040 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456819de70 con 0x7f45681096a0 2024-09-12T02:55:24.452 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.981+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a0b60 con 0x7f45681096a0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.982+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4558045070 con 0x7f45681096a0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.982+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456810ddd0 con 0x7f45681096a0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.983+0000 7f4564c00640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 0x7f454c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.983+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f455806c0f0 con 0x7f45681096a0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.983+0000 7f455ea00640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 0x7f454c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.984+0000 7f455ea00640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 0x7f454c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f4550006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:18.988+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f455806ba40 con 0x7f45681096a0 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:19.179+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi057", "addr": "172.21.15.57", "target": ["mon-mgr", ""]}) -- 0x7f456810c1d0 con 0x7f454c038f70 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.741+0000 7f4564c00640 1 -- 172.21.15.57:0/2963991181 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f456810c1d0 con 0x7f454c038f70 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 msgr2=0x7f454c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 0x7f454c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f4550006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:24.453 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 msgr2=0x7f456819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4558009940 tx=0x7f4558032ce0 comp rx=0 tx=0).stop 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 shutdown_connections 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f454c038f70 0x7f454c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 --2- 172.21.15.57:0/2963991181 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f45681096a0 0x7f456819d6a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.744+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 >> 172.21.15.57:0/2963991181 conn(0x7f456807cd10 msgr2=0x7f456810bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.745+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 shutdown_connections 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:23.745+0000 7f456d000640 1 -- 172.21.15.57:0/2963991181 wait complete. 2024-09-12T02:55:24.454 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mon service... 2024-09-12T02:55:25.071 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:24 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:25.071 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:24 smithi057 ceph-mon[30361]: Added host smithi057 2024-09-12T02:55:25.072 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:24 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:26.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:26 smithi057 ceph-mon[30361]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:26.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:26 smithi057 ceph-mon[30361]: Saving service mon spec with placement count:5 2024-09-12T02:55:26.601 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:26 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:26.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T02:55:26.963 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.070+0000 7fb7ae000640 1 Processor -- start 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.070+0000 7fb7ae000640 1 -- start start 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7ae000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7ae000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8108070 con 0x7fb7a81076a0 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7a7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7a7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41968/0 (socket says 172.21.15.57:41968) 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7a7400640 1 -- 172.21.15.57:0/3208286039 learned_addr learned my addr 172.21.15.57:0/3208286039 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.071+0000 7fb7a7400640 1 -- 172.21.15.57:0/3208286039 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a8108850 con 0x7fb7a81076a0 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.072+0000 7fb7a7400640 1 --2- 172.21.15.57:0/3208286039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb798009820 tx=0x7fb79802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1871b3bf40056a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7a6000640 1 -- 172.21.15.57:0/3208286039 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb798037070 con 0x7fb7a81076a0 2024-09-12T02:55:26.964 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7a6000640 1 -- 172.21.15.57:0/3208286039 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb798032900 con 0x7fb7a81076a0 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7a6000640 1 -- 172.21.15.57:0/3208286039 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb798030850 con 0x7fb7a81076a0 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 -- 172.21.15.57:0/3208286039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 msgr2=0x7fb7a8107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 --2- 172.21.15.57:0/3208286039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb798009820 tx=0x7fb79802a270 comp rx=0 tx=0).stop 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 -- 172.21.15.57:0/3208286039 shutdown_connections 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 --2- 172.21.15.57:0/3208286039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a8107aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 -- 172.21.15.57:0/3208286039 >> 172.21.15.57:0/3208286039 conn(0x7fb7a8102e30 msgr2=0x7fb7a8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 -- 172.21.15.57:0/3208286039 shutdown_connections 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.073+0000 7fb7ae000640 1 -- 172.21.15.57:0/3208286039 wait complete. 2024-09-12T02:55:26.965 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.074+0000 7fb7ae000640 1 Processor -- start 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7ae000640 1 -- start start 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7ae000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7ae000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a810bc00 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7a7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7a7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41984/0 (socket says 172.21.15.57:41984) 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7a7400640 1 -- 172.21.15.57:0/4167412579 learned_addr learned my addr 172.21.15.57:0/4167412579 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.075+0000 7fb7a7400640 1 -- 172.21.15.57:0/4167412579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a819ddf0 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb7a7400640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7980097f0 tx=0x7fb798030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb798037040 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a819e080 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb798040070 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb79803a390 con 0x7fb7a81076a0 2024-09-12T02:55:26.966 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.076+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a81a0d70 con 0x7fb7a81076a0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.077+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb798032900 con 0x7fb7a81076a0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.077+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a8107b20 con 0x7fb7a81076a0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.078+0000 7fb79fe00640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 0x7fb78c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.078+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb79806c540 con 0x7fb7a81076a0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.078+0000 7fb7a6a00640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 0x7fb78c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.079+0000 7fb7a6a00640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 0x7fb78c03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098c0 tx=0x7fb794006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.083+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb79806be90 con 0x7fb7a81076a0 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.312+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb7a8104090 con 0x7fb78c038f70 2024-09-12T02:55:26.967 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.323+0000 7fb79fe00640 1 -- 172.21.15.57:0/4167412579 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb7a8104090 con 0x7fb78c038f70 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 msgr2=0x7fb78c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 0x7fb78c03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb7940098c0 tx=0x7fb794006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 msgr2=0x7fb7a819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7980097f0 tx=0x7fb798030d80 comp rx=0 tx=0).stop 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 shutdown_connections 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fb78c038f70 0x7fb78c03b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 --2- 172.21.15.57:0/4167412579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb7a81076a0 0x7fb7a819d8b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 >> 172.21.15.57:0/4167412579 conn(0x7fb7a8102e30 msgr2=0x7fb7a8103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 shutdown_connections 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:25.325+0000 7fb7ae000640 1 -- 172.21.15.57:0/4167412579 wait complete. 2024-09-12T02:55:26.968 INFO:teuthology.orchestra.run.smithi057.stdout:Deploying unmanaged mgr service... 2024-09-12T02:55:27.782 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:27 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:27.783 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:27 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.500+0000 7fe01be00640 1 Processor -- start 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe01be00640 1 -- start start 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe01be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe01be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410a250 con 0x7fe014109880 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe019400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe019400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41998/0 (socket says 172.21.15.57:41998) 2024-09-12T02:55:28.654 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.501+0000 7fe019400640 1 -- 172.21.15.57:0/2165093910 learned_addr learned my addr 172.21.15.57:0/2165093910 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.502+0000 7fe019400640 1 -- 172.21.15.57:0/2165093910 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01410aa80 con 0x7fe014109880 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.503+0000 7fe019400640 1 --2- 172.21.15.57:0/2165093910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe008009820 tx=0x7fe00802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304510e94ba1e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.503+0000 7fe013e00640 1 -- 172.21.15.57:0/2165093910 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008037070 con 0x7fe014109880 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.503+0000 7fe013e00640 1 -- 172.21.15.57:0/2165093910 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe008032920 con 0x7fe014109880 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.503+0000 7fe013e00640 1 -- 172.21.15.57:0/2165093910 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008030810 con 0x7fe014109880 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 -- 172.21.15.57:0/2165093910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 msgr2=0x7fe014109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 --2- 172.21.15.57:0/2165093910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe008009820 tx=0x7fe00802a270 comp rx=0 tx=0).stop 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 -- 172.21.15.57:0/2165093910 shutdown_connections 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 --2- 172.21.15.57:0/2165093910 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe014109c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 -- 172.21.15.57:0/2165093910 >> 172.21.15.57:0/2165093910 conn(0x7fe01407ce60 msgr2=0x7fe01407d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 -- 172.21.15.57:0/2165093910 shutdown_connections 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.504+0000 7fe01be00640 1 -- 172.21.15.57:0/2165093910 wait complete. 2024-09-12T02:55:28.655 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.505+0000 7fe01be00640 1 Processor -- start 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.505+0000 7fe01be00640 1 -- start start 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe01be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe01be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe01410de30 con 0x7fe014109880 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe019400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe019400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42004/0 (socket says 172.21.15.57:42004) 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe019400640 1 -- 172.21.15.57:0/341056953 learned_addr learned my addr 172.21.15.57:0/341056953 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.506+0000 7fe019400640 1 -- 172.21.15.57:0/341056953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01419de10 con 0x7fe014109880 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe019400640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe008030560 tx=0x7fe008032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008037040 con 0x7fe014109880 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe008040070 con 0x7fe014109880 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe00803a740 con 0x7fe014109880 2024-09-12T02:55:28.656 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01419e0a0 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.507+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0141a0d90 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.508+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe00803b420 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.508+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01410dfb0 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.509+0000 7fe012000640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 0x7fdffc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.509+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe00806c650 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.509+0000 7fe018a00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 0x7fdffc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.510+0000 7fe018a00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 0x7fdffc03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098c0 tx=0x7fe000006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.519+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe00804a200 con 0x7fe014109880 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.782+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe014107410 con 0x7fdffc038f70 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.787+0000 7fe012000640 1 -- 172.21.15.57:0/341056953 <== mgr.14118 v2:172.21.15.57:6800/522425633 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe014107410 con 0x7fdffc038f70 2024-09-12T02:55:28.657 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 msgr2=0x7fdffc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 0x7fdffc03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0000098c0 tx=0x7fe000006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 msgr2=0x7fe01419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe008030560 tx=0x7fe008032670 comp rx=0 tx=0).stop 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 shutdown_connections 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fdffc038f70 0x7fdffc03b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 --2- 172.21.15.57:0/341056953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe014109880 0x7fe01419d8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 >> 172.21.15.57:0/341056953 conn(0x7fe01407ce60 msgr2=0x7fe014106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.789+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 shutdown_connections 2024-09-12T02:55:28.658 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:27.790+0000 7fe01be00640 1 -- 172.21.15.57:0/341056953 wait complete. 2024-09-12T02:55:28.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:28 smithi057 ceph-mon[30361]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:28.928 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:28 smithi057 ceph-mon[30361]: Saving service mgr spec with placement count:2 2024-09-12T02:55:28.929 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:28 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:30.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.646+0000 7fbcc5800640 1 Processor -- start 2024-09-12T02:55:30.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.646+0000 7fbcc5800640 1 -- start start 2024-09-12T02:55:30.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcc5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcc5800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010b910 con 0x7fbcc010af40 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46700/0 (socket says 172.21.15.57:46700) 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcbea00640 1 -- 172.21.15.57:0/3012132200 learned_addr learned my addr 172.21.15.57:0/3012132200 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.647+0000 7fbcbea00640 1 -- 172.21.15.57:0/3012132200 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc010ba90 con 0x7fbcc010af40 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.648+0000 7fbcbea00640 1 --2- 172.21.15.57:0/3012132200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009820 tx=0x7fbcb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12aee9a7682095bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcbd600640 1 -- 172.21.15.57:0/3012132200 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb0037070 con 0x7fbcc010af40 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcbd600640 1 -- 172.21.15.57:0/3012132200 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb0032920 con 0x7fbcc010af40 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcbd600640 1 -- 172.21.15.57:0/3012132200 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb0030810 con 0x7fbcc010af40 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcc5800640 1 -- 172.21.15.57:0/3012132200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 msgr2=0x7fbcc010b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcc5800640 1 --2- 172.21.15.57:0/3012132200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009820 tx=0x7fbcb002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcc5800640 1 -- 172.21.15.57:0/3012132200 shutdown_connections 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcc5800640 1 --2- 172.21.15.57:0/3012132200 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc010b340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.442 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.649+0000 7fbcc5800640 1 -- 172.21.15.57:0/3012132200 >> 172.21.15.57:0/3012132200 conn(0x7fbcc007d1e0 msgr2=0x7fbcc007d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.650+0000 7fbcc5800640 1 -- 172.21.15.57:0/3012132200 shutdown_connections 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.650+0000 7fbcc5800640 1 -- 172.21.15.57:0/3012132200 wait complete. 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcc5800640 1 Processor -- start 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcc5800640 1 -- start start 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcc5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcc5800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc010dec0 con 0x7fbcc010af40 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcbea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46708/0 (socket says 172.21.15.57:46708) 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.651+0000 7fbcbea00640 1 -- 172.21.15.57:0/827530916 learned_addr learned my addr 172.21.15.57:0/827530916 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.652+0000 7fbcbea00640 1 -- 172.21.15.57:0/827530916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc019df50 con 0x7fbcc010af40 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.652+0000 7fbcbea00640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097f0 tx=0x7fbcb00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.652+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb0037040 con 0x7fbcc010af40 2024-09-12T02:55:30.443 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.652+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb0030cc0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.652+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc019e1e0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.653+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcb003a640 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.653+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a0ed0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.654+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc010b3c0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.654+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbcb003b3f0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.654+0000 7fbcb7400640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 0x7fbca403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.655+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbcb006c510 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.655+0000 7fbcbe000640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 0x7fbca403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.655+0000 7fbcbe000640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 0x7fbca403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0098c0 tx=0x7fbcac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.660+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcb00328c0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.850+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbcc01a13c0 con 0x7fbcc010af40 2024-09-12T02:55:30.444 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.855+0000 7fbcb7400640 1 -- 172.21.15.57:0/827530916 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbcb0042dc0 con 0x7fbcc010af40 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 msgr2=0x7fbca403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 0x7fbca403b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0098c0 tx=0x7fbcac006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 msgr2=0x7fbcc019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097f0 tx=0x7fbcb00327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 shutdown_connections 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fbca4038fc0 0x7fbca403b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 --2- 172.21.15.57:0/827530916 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbcc010af40 0x7fbcc019da10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 >> 172.21.15.57:0/827530916 conn(0x7fbcc007d1e0 msgr2=0x7fbcc0107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.857+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 shutdown_connections 2024-09-12T02:55:30.445 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:29.858+0000 7fbcc5800640 1 -- 172.21.15.57:0/827530916 wait complete. 2024-09-12T02:55:30.724 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:30 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:30.724 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:30 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/827530916' entity='client.admin' 2024-09-12T02:55:31.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:31 smithi057 ceph-mon[30361]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:31.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:31 smithi057 ceph-mon[30361]: from='mgr.14118 172.21.15.57:0/3036419448' entity='mgr.x' 2024-09-12T02:55:31.834 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:31 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2180005371' entity='client.admin' 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.976+0000 7f0e1f800640 1 Processor -- start 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1f800640 1 -- start start 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1f800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1f800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810b910 con 0x7f0e1810af40 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46714/0 (socket says 172.21.15.57:46714) 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.977+0000 7f0e1ce00640 1 -- 172.21.15.57:0/2993696075 learned_addr learned my addr 172.21.15.57:0/2993696075 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.978+0000 7f0e1ce00640 1 -- 172.21.15.57:0/2993696075 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1810ba90 con 0x7f0e1810af40 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.979+0000 7f0e1ce00640 1 --2- 172.21.15.57:0/2993696075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009820 tx=0x7f0e0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff283aab57aa0cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:31.844 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.979+0000 7f0e17400640 1 -- 172.21.15.57:0/2993696075 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e0c037070 con 0x7f0e1810af40 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.979+0000 7f0e17400640 1 -- 172.21.15.57:0/2993696075 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c032920 con 0x7f0e1810af40 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.979+0000 7f0e17400640 1 -- 172.21.15.57:0/2993696075 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e0c030810 con 0x7f0e1810af40 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 -- 172.21.15.57:0/2993696075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 msgr2=0x7f0e1810b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2993696075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009820 tx=0x7f0e0c02a270 comp rx=0 tx=0).stop 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 -- 172.21.15.57:0/2993696075 shutdown_connections 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2993696075 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1810b340 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 -- 172.21.15.57:0/2993696075 >> 172.21.15.57:0/2993696075 conn(0x7f0e1807d1e0 msgr2=0x7f0e1807d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 -- 172.21.15.57:0/2993696075 shutdown_connections 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.980+0000 7f0e1f800640 1 -- 172.21.15.57:0/2993696075 wait complete. 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.981+0000 7f0e1f800640 1 Processor -- start 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.981+0000 7f0e1f800640 1 -- start start 2024-09-12T02:55:31.845 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.981+0000 7f0e1f800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.981+0000 7f0e1f800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810dec0 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.982+0000 7f0e1ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.982+0000 7f0e1ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46726/0 (socket says 172.21.15.57:46726) 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.982+0000 7f0e1ce00640 1 -- 172.21.15.57:0/2180005371 learned_addr learned my addr 172.21.15.57:0/2180005371 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.982+0000 7f0e1ce00640 1 -- 172.21.15.57:0/2180005371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1819df50 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.982+0000 7f0e1ce00640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097f0 tx=0x7f0e0c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.983+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e0c037040 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.983+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c030cc0 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.983+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1819e1e0 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.983+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e0c03a640 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.983+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181a0ed0 con 0x7f0e1810af40 2024-09-12T02:55:31.846 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.984+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1810b3c0 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.985+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0e0c03b3f0 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.985+0000 7f0e15600640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 0x7f0e0003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.985+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e0c06c590 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.985+0000 7f0e17e00640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 0x7f0e0003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.986+0000 7f0e17e00640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 0x7f0e0003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e040098c0 tx=0x7f0e04006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:30.990+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e0c0328c0 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.187+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0e181a13c0 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.190+0000 7f0e15600640 1 -- 172.21.15.57:0/2180005371 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0e0c06bee0 con 0x7f0e1810af40 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 msgr2=0x7f0e0003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 0x7f0e0003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0e040098c0 tx=0x7f0e04006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 msgr2=0x7f0e1819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.847 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097f0 tx=0x7f0e0c0327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 shutdown_connections 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f0e00038fc0 0x7f0e0003b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 --2- 172.21.15.57:0/2180005371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e1810af40 0x7f0e1819da10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.192+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 >> 172.21.15.57:0/2180005371 conn(0x7f0e1807d1e0 msgr2=0x7f0e18107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.193+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 shutdown_connections 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:31.193+0000 7f0e1f800640 1 -- 172.21.15.57:0/2180005371 wait complete. 2024-09-12T02:55:31.848 INFO:teuthology.orchestra.run.smithi057.stdout:Enabling the dashboard module... 2024-09-12T02:55:33.746 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-mon[30361]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:55:33.746 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/752546397' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T02:55:33.747 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-12T02:55:33.747 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-12T02:55:33.747 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:33.745+0000 7f50b2409040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:55:34.009 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:33.818+0000 7f50b2409040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.508+0000 7f8f13e00640 1 Processor -- start 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.508+0000 7f8f13e00640 1 -- start start 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.508+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.509+0000 7f8f13e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10a300 con 0x7f8f0c109930 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.509+0000 7f8f11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:34.182 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.509+0000 7f8f11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46742/0 (socket says 172.21.15.57:46742) 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.509+0000 7f8f11400640 1 -- 172.21.15.57:0/1716462546 learned_addr learned my addr 172.21.15.57:0/1716462546 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.509+0000 7f8f11400640 1 -- 172.21.15.57:0/1716462546 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c10aae0 con 0x7f8f0c109930 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.510+0000 7f8f11400640 1 --2- 172.21.15.57:0/1716462546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009820 tx=0x7f8f0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311200a665b1558 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.510+0000 7f8f0be00640 1 -- 172.21.15.57:0/1716462546 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f00037070 con 0x7f8f0c109930 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f0be00640 1 -- 172.21.15.57:0/1716462546 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00032920 con 0x7f8f0c109930 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f0be00640 1 -- 172.21.15.57:0/1716462546 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f00030810 con 0x7f8f0c109930 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 -- 172.21.15.57:0/1716462546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 msgr2=0x7f8f0c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 --2- 172.21.15.57:0/1716462546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009820 tx=0x7f8f0002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 -- 172.21.15.57:0/1716462546 shutdown_connections 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 --2- 172.21.15.57:0/1716462546 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c109d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 -- 172.21.15.57:0/1716462546 >> 172.21.15.57:0/1716462546 conn(0x7f8f0c07cf00 msgr2=0x7f8f0c07d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 -- 172.21.15.57:0/1716462546 shutdown_connections 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.511+0000 7f8f13e00640 1 -- 172.21.15.57:0/1716462546 wait complete. 2024-09-12T02:55:34.183 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.512+0000 7f8f13e00640 1 Processor -- start 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.512+0000 7f8f13e00640 1 -- start start 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f13e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f13e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10de90 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f11400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46754/0 (socket says 172.21.15.57:46754) 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f11400640 1 -- 172.21.15.57:0/752546397 learned_addr learned my addr 172.21.15.57:0/752546397 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f11400640 1 -- 172.21.15.57:0/752546397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c19de20 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.513+0000 7f8f11400640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097f0 tx=0x7f8f000327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.514+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f00037040 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.514+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f00030cc0 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.514+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f0003a640 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.514+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c19e0b0 con 0x7f8f0c109930 2024-09-12T02:55:34.184 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.514+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c1a0da0 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.515+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8f0003b3f0 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.515+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0c109db0 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.515+0000 7f8f0a000640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 0x7f8ef403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.515+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f0006c590 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.516+0000 7f8f10a00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 0x7f8ef403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.516+0000 7f8f10a00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 0x7f8ef403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80098c0 tx=0x7f8ef8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.521+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f000313a0 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:32.771+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f8f0c1a1390 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.486+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8f00038250 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.487+0000 7f8f0a000640 1 -- 172.21.15.57:0/752546397 <== mon.0 v2:172.21.15.57:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8f00041d10 con 0x7f8f0c109930 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 msgr2=0x7f8ef403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:34.185 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 0x7f8ef403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80098c0 tx=0x7f8ef8006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 msgr2=0x7f8f0c19d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097f0 tx=0x7f8f000327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 shutdown_connections 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f8ef4038fc0 0x7f8ef403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 --2- 172.21.15.57:0/752546397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8f0c109930 0x7f8f0c19d8e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.489+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 >> 172.21.15.57:0/752546397 conn(0x7f8f0c07cf00 msgr2=0x7f8f0c106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.490+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 shutdown_connections 2024-09-12T02:55:34.186 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:33.490+0000 7f8f13e00640 1 -- 172.21.15.57:0/752546397 wait complete. 2024-09-12T02:55:34.273 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.008+0000 7f50b2409040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:55:34.274 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.092+0000 7f50b2409040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:55:34.274 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.272+0000 7f50b2409040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:34.537 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/752546397' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T02:55:34.538 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-mon[30361]: mgrmap e9: x(active, since 23s) 2024-09-12T02:55:34.538 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.445+0000 7f50b2409040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:55:35.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.835+0000 7f50b2409040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:55:35.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:34.911+0000 7f50b2409040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:55:35.458 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:35.146+0000 7f50b2409040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:55:35.708 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.748+0000 7fa1f6e00640 1 Processor -- start 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.748+0000 7fa1f6e00640 1 -- start start 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.748+0000 7fa1f6e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.748+0000 7fa1f6e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010a250 con 0x7fa1f0109880 2024-09-12T02:55:35.709 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.749+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.749+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46786/0 (socket says 172.21.15.57:46786) 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.749+0000 7fa1efe00640 1 -- 172.21.15.57:0/1121316019 learned_addr learned my addr 172.21.15.57:0/1121316019 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.749+0000 7fa1efe00640 1 -- 172.21.15.57:0/1121316019 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f010aa80 con 0x7fa1f0109880 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.750+0000 7fa1efe00640 1 --2- 172.21.15.57:0/1121316019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009820 tx=0x7fa1e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42a22b623f4f732 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.750+0000 7fa1eea00640 1 -- 172.21.15.57:0/1121316019 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0037070 con 0x7fa1f0109880 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.750+0000 7fa1eea00640 1 -- 172.21.15.57:0/1121316019 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0032920 con 0x7fa1f0109880 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.750+0000 7fa1eea00640 1 -- 172.21.15.57:0/1121316019 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0030810 con 0x7fa1f0109880 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 -- 172.21.15.57:0/1121316019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 msgr2=0x7fa1f0109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/1121316019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009820 tx=0x7fa1e002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 -- 172.21.15.57:0/1121316019 shutdown_connections 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/1121316019 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f0109c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 -- 172.21.15.57:0/1121316019 >> 172.21.15.57:0/1121316019 conn(0x7fa1f007ce60 msgr2=0x7fa1f007d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:35.710 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.751+0000 7fa1f6e00640 1 -- 172.21.15.57:0/1121316019 shutdown_connections 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.752+0000 7fa1f6e00640 1 -- 172.21.15.57:0/1121316019 wait complete. 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.752+0000 7fa1f6e00640 1 Processor -- start 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1f6e00640 1 -- start start 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1f6e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1f6e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f010de30 con 0x7fa1f0109880 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46790/0 (socket says 172.21.15.57:46790) 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.753+0000 7fa1efe00640 1 -- 172.21.15.57:0/564560094 learned_addr learned my addr 172.21.15.57:0/564560094 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1efe00640 1 -- 172.21.15.57:0/564560094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f019de10 con 0x7fa1f0109880 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1efe00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00097f0 tx=0x7fa1e00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e0037040 con 0x7fa1f0109880 2024-09-12T02:55:35.711 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0030cc0 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f019e0a0 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.754+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1e003a640 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.755+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a0d90 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.756+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa1e003b3f0 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.756+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f010dfb0 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.756+0000 7fa1ecc00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 0x7fa1d403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.756+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1e006c0f0 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.757+0000 7fa1ef400640 1 -- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 msgr2=0x7fa1d403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.757+0000 7fa1ef400640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 0x7fa1d403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.762+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1e0031e50 con 0x7fa1f0109880 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.958+0000 7fa1ef400640 1 -- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 msgr2=0x7fa1d403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:35.712 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.958+0000 7fa1ef400640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 0x7fa1d403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.995+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa1f01a0f60 con 0x7fa1f0109880 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.996+0000 7fa1ecc00640 1 -- 172.21.15.57:0/564560094 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa1e006ba40 con 0x7fa1f0109880 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 msgr2=0x7fa1d403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 0x7fa1d403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 msgr2=0x7fa1f019d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00097f0 tx=0x7fa1e00327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 shutdown_connections 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7fa1d4039020 0x7fa1d403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 --2- 172.21.15.57:0/564560094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa1f0109880 0x7fa1f019d8d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 >> 172.21.15.57:0/564560094 conn(0x7fa1f007ce60 msgr2=0x7fa1f0106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 shutdown_connections 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:34.998+0000 7fa1f6e00640 1 -- 172.21.15.57:0/564560094 wait complete. 2024-09-12T02:55:35.713 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for the mgr to restart... 2024-09-12T02:55:35.714 INFO:teuthology.orchestra.run.smithi057.stdout:Waiting for mgr epoch 9... 2024-09-12T02:55:35.848 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:35 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/564560094' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:55:36.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:36.325+0000 7f50b2409040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:55:36.827 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:36.410+0000 7f50b2409040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:55:37.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:37 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:37.353+0000 7f50b2409040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:55:37.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:37 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:37.494+0000 7f50b2409040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:55:37.916 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:37 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:37.663+0000 7f50b2409040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:55:37.917 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:37 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:37.738+0000 7f50b2409040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:55:37.917 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:37 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:37.915+0000 7f50b2409040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:55:38.262 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.000+0000 7f50b2409040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:55:38.262 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.074+0000 7f50b2409040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:55:38.262 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.261+0000 7f50b2409040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:55:38.523 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.341+0000 7f50b2409040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:55:38.523 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.422+0000 7f50b2409040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:55:38.523 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.522+0000 7f50b2409040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:55:38.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.595+0000 7f50b2409040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:55:38.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:38 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:38.668+0000 7f50b2409040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:55:39.869 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:39.616+0000 7f50b2409040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: Active manager daemon x restarted 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: Activating manager daemon x 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: mgrmap e10: x(active, starting, since 0.00556514s) 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:55:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:55:39.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:55:39.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:55:39.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:55:39.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: Manager daemon x is now available 2024-09-12T02:55:39.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:41.044 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:55:41.044 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:55:41.044 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:40 smithi057 ceph-mon[30361]: mgrmap e11: x(active, since 1.00903s) 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.216+0000 7f2d31200640 1 Processor -- start 2024-09-12T02:55:41.332 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d31200640 1 -- start start 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d31200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d31200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c106d80 con 0x7f2d2c1063b0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d2b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d2b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46792/0 (socket says 172.21.15.57:46792) 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.217+0000 7f2d2b400640 1 -- 172.21.15.57:0/1682092782 learned_addr learned my addr 172.21.15.57:0/1682092782 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.218+0000 7f2d2b400640 1 -- 172.21.15.57:0/1682092782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1075b0 con 0x7f2d2c1063b0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.219+0000 7f2d2b400640 1 --2- 172.21.15.57:0/1682092782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009820 tx=0x7f2d1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c3174201757fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.219+0000 7f2d2aa00640 1 -- 172.21.15.57:0/1682092782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d1c037070 con 0x7f2d2c1063b0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.219+0000 7f2d2aa00640 1 -- 172.21.15.57:0/1682092782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c032920 con 0x7f2d2c1063b0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.219+0000 7f2d2aa00640 1 -- 172.21.15.57:0/1682092782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d1c030810 con 0x7f2d2c1063b0 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 -- 172.21.15.57:0/1682092782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 msgr2=0x7f2d2c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.333 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 --2- 172.21.15.57:0/1682092782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009820 tx=0x7f2d1c02a270 comp rx=0 tx=0).stop 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 -- 172.21.15.57:0/1682092782 shutdown_connections 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 --2- 172.21.15.57:0/1682092782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1067b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 -- 172.21.15.57:0/1682092782 >> 172.21.15.57:0/1682092782 conn(0x7f2d2c101b40 msgr2=0x7f2d2c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 -- 172.21.15.57:0/1682092782 shutdown_connections 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.220+0000 7f2d31200640 1 -- 172.21.15.57:0/1682092782 wait complete. 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.221+0000 7f2d31200640 1 Processor -- start 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.221+0000 7f2d31200640 1 -- start start 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d31200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d31200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c07b0d0 con 0x7f2d2c1063b0 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d2b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d2b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46794/0 (socket says 172.21.15.57:46794) 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d2b400640 1 -- 172.21.15.57:0/4163516590 learned_addr learned my addr 172.21.15.57:0/4163516590 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:41.334 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.222+0000 7f2d2b400640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1997f0 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d2b400640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c030560 tx=0x7f2d1c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d1c037040 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c040070 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c199a80 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d1c03a740 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.223+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c19c770 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.224+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2d1c03b420 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.224+0000 7f2d28c00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.224+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2d1003ffb0 con 0x7f2d1003d3e0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.225+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d1c06c6b0 con 0x7f2d2c1063b0 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.225+0000 7f2d23e00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:41.335 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.225+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.426+0000 7f2d23e00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.426+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.827+0000 7f2d23e00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:36.827+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:37.628+0000 7f2d23e00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:37.628+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:39.230+0000 7f2d23e00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/522425633 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:39.230+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:39.622+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f2d1c03a9b0 con 0x7f2d2c1063b0 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:39.622+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 msgr2=0x7f2d1003f8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:39.622+0000 7f2d28c00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.336 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.625+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f2d1c03bb90 con 0x7f2d2c1063b0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.625+0000 7f2d28c00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 0x7f2d100432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.625+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2d1c03de70 con 0x7f2d10040ef0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.626+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 0x7f2d100432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.627+0000 7f2d23e00640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 0x7f2d100432e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003980 tx=0x7f2d14009290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.628+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f2d1c03de70 con 0x7f2d10040ef0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.635+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2d2c1067b0 con 0x7f2d10040ef0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d28c00640 1 -- 172.21.15.57:0/4163516590 <== mgr.14150 v2:172.21.15.57:6800/480086157 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f2d2c1067b0 con 0x7f2d10040ef0 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 msgr2=0x7f2d100432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 0x7f2d100432e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003980 tx=0x7f2d14009290 comp rx=0 tx=0).stop 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 msgr2=0x7f2d2c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c030560 tx=0x7f2d1c032670 comp rx=0 tx=0).stop 2024-09-12T02:55:41.337 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 shutdown_connections 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2d10040ef0 0x7f2d100432e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:6800/522425633,v1:172.21.15.57:6801/522425633] conn(0x7f2d1003d3e0 0x7f2d1003f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 --2- 172.21.15.57:0/4163516590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2d2c1063b0 0x7f2d2c1992b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.636+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 >> 172.21.15.57:0/4163516590 conn(0x7f2d2c101b40 msgr2=0x7f2d2c18f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.637+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 shutdown_connections 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:40.637+0000 7f2d31200640 1 -- 172.21.15.57:0/4163516590 wait complete. 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:mgr epoch 9 is available 2024-09-12T02:55:41.338 INFO:teuthology.orchestra.run.smithi057.stdout:Generating a dashboard self-signed certificate... 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:41] ENGINE Bus STARTING 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:41] ENGINE Serving on https://172.21.15.57:7150 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:41] ENGINE Error in HTTPServer.serve 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:55:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:55:41.871 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:41.572+0000 7f500ea00640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:55:41] ENGINE Error in HTTPServer.serve 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Traceback (most recent call last): 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._run(expiration_interval) 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:55:41.872 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.do_handshake() 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:55:41.873 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:55:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:55:42.843 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:41] ENGINE Serving on http://172.21.15.57:8765 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: [12/Sep/2024:02:55:41] ENGINE Bus STARTED 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: mgrmap e12: x(active, since 2s) 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:42.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.806+0000 7f9750c00640 1 Processor -- start 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.807+0000 7f9750c00640 1 -- start start 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.807+0000 7f9750c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.807+0000 7f9750c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10a090 con 0x7f974c1096c0 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.807+0000 7f974b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.808+0000 7f974b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45330/0 (socket says 172.21.15.57:45330) 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.808+0000 7f974b400640 1 -- 172.21.15.57:0/3267999429 learned_addr learned my addr 172.21.15.57:0/3267999429 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.808+0000 7f974b400640 1 -- 172.21.15.57:0/3267999429 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c10a8c0 con 0x7f974c1096c0 2024-09-12T02:55:42.857 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.809+0000 7f974b400640 1 --2- 172.21.15.57:0/3267999429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f973c009820 tx=0x7f973c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3194d0817bb74017 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.809+0000 7f974aa00640 1 -- 172.21.15.57:0/3267999429 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c037070 con 0x7f974c1096c0 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.809+0000 7f974aa00640 1 -- 172.21.15.57:0/3267999429 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973c032920 con 0x7f974c1096c0 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.809+0000 7f974aa00640 1 -- 172.21.15.57:0/3267999429 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c030810 con 0x7f974c1096c0 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 -- 172.21.15.57:0/3267999429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 msgr2=0x7f974c109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 --2- 172.21.15.57:0/3267999429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f973c009820 tx=0x7f973c02a270 comp rx=0 tx=0).stop 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 -- 172.21.15.57:0/3267999429 shutdown_connections 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 --2- 172.21.15.57:0/3267999429 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c109ac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 -- 172.21.15.57:0/3267999429 >> 172.21.15.57:0/3267999429 conn(0x7f974c07ccc0 msgr2=0x7f974c07d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 -- 172.21.15.57:0/3267999429 shutdown_connections 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.810+0000 7f9750c00640 1 -- 172.21.15.57:0/3267999429 wait complete. 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.811+0000 7f9750c00640 1 Processor -- start 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.811+0000 7f9750c00640 1 -- start start 2024-09-12T02:55:42.858 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.811+0000 7f9750c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.811+0000 7f9750c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f974c10dc70 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.812+0000 7f974b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.812+0000 7f974b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45336/0 (socket says 172.21.15.57:45336) 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.812+0000 7f974b400640 1 -- 172.21.15.57:0/2520610968 learned_addr learned my addr 172.21.15.57:0/2520610968 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.812+0000 7f974b400640 1 -- 172.21.15.57:0/2520610968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19dc00 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.812+0000 7f974b400640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f973c0097f0 tx=0x7f973c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.813+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c037040 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.813+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973c030cc0 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.813+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f973c03a640 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.813+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19de90 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.813+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a0b80 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.814+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974c10ddf0 con 0x7f974c1096c0 2024-09-12T02:55:42.859 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.814+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f973c03b3f0 con 0x7f974c1096c0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.815+0000 7f9748c00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 0x7f973003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.815+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f973c06c6b0 con 0x7f974c1096c0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.815+0000 7f9743e00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 0x7f973003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.815+0000 7f9743e00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 0x7f973003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97340098c0 tx=0x7f9734006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:41.820+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f973c042e10 con 0x7f974c1096c0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.012+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f974c10c1f0 con 0x7f9730038fc0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.099+0000 7f9748c00640 1 -- 172.21.15.57:0/2520610968 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f974c10c1f0 con 0x7f9730038fc0 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 msgr2=0x7f973003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 0x7f973003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f97340098c0 tx=0x7f9734006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 msgr2=0x7f974c19d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f973c0097f0 tx=0x7f973c0327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 shutdown_connections 2024-09-12T02:55:42.860 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9730038fc0 0x7f973003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.861 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 --2- 172.21.15.57:0/2520610968 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f974c1096c0 0x7f974c19d6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.861 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 >> 172.21.15.57:0/2520610968 conn(0x7f974c07ccc0 msgr2=0x7f974c10bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:42.861 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 shutdown_connections 2024-09-12T02:55:42.861 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:42.102+0000 7f9750c00640 1 -- 172.21.15.57:0/2520610968 wait complete. 2024-09-12T02:55:42.861 INFO:teuthology.orchestra.run.smithi057.stdout:Creating initial admin user... 2024-09-12T02:55:44.579 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ynlj06KBZyMlsNRXxf5lfu4MC1SIX4FVnf1L.uO5as9AcXnOMyvLG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726109743, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T02:55:44.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.418+0000 7f7e2ac00640 1 Processor -- start 2024-09-12T02:55:44.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e2ac00640 1 -- start start 2024-09-12T02:55:44.580 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e2ac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:44.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e2ac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2410a250 con 0x7f7e24109880 2024-09-12T02:55:44.586 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45346/0 (socket says 172.21.15.57:45346) 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.419+0000 7f7e23e00640 1 -- 172.21.15.57:0/3570466583 learned_addr learned my addr 172.21.15.57:0/3570466583 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.420+0000 7f7e23e00640 1 -- 172.21.15.57:0/3570466583 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2410aa80 con 0x7f7e24109880 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.421+0000 7f7e23e00640 1 --2- 172.21.15.57:0/3570466583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009820 tx=0x7f7e1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698160ec34007caf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.421+0000 7f7e22a00640 1 -- 172.21.15.57:0/3570466583 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e14037070 con 0x7f7e24109880 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.421+0000 7f7e22a00640 1 -- 172.21.15.57:0/3570466583 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e14032920 con 0x7f7e24109880 2024-09-12T02:55:44.587 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.421+0000 7f7e22a00640 1 -- 172.21.15.57:0/3570466583 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e14030810 con 0x7f7e24109880 2024-09-12T02:55:44.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3570466583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 msgr2=0x7f7e24109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:44.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3570466583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009820 tx=0x7f7e1402a270 comp rx=0 tx=0).stop 2024-09-12T02:55:44.588 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3570466583 shutdown_connections 2024-09-12T02:55:44.589 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3570466583 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e24109c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:44.589 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3570466583 >> 172.21.15.57:0/3570466583 conn(0x7f7e2407ce60 msgr2=0x7f7e2407d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:44.589 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3570466583 shutdown_connections 2024-09-12T02:55:44.589 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.422+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3570466583 wait complete. 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.423+0000 7f7e2ac00640 1 Processor -- start 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.423+0000 7f7e2ac00640 1 -- start start 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.423+0000 7f7e2ac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.423+0000 7f7e2ac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2410de30 con 0x7f7e24109880 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.424+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:44.590 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.424+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45350/0 (socket says 172.21.15.57:45350) 2024-09-12T02:55:44.591 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.424+0000 7f7e23e00640 1 -- 172.21.15.57:0/3221729094 learned_addr learned my addr 172.21.15.57:0/3221729094 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:44.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.424+0000 7f7e23e00640 1 -- 172.21.15.57:0/3221729094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2419de10 con 0x7f7e24109880 2024-09-12T02:55:44.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.424+0000 7f7e23e00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7e140097f0 tx=0x7f7e140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:44.596 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.425+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e14037040 con 0x7f7e24109880 2024-09-12T02:55:44.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.425+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e14030cc0 con 0x7f7e24109880 2024-09-12T02:55:44.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.425+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1403a640 con 0x7f7e24109880 2024-09-12T02:55:44.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.425+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2419e0a0 con 0x7f7e24109880 2024-09-12T02:55:44.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.425+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a0d90 con 0x7f7e24109880 2024-09-12T02:55:44.597 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.426+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7e1403b3f0 con 0x7f7e24109880 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.426+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2410dfb0 con 0x7f7e24109880 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.427+0000 7f7e20c00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 0x7f7e0803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.427+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e1406c6d0 con 0x7f7e24109880 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.427+0000 7f7e23400640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 0x7f7e0803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.428+0000 7f7e23400640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 0x7f7e0803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e100098c0 tx=0x7f7e10006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:44.598 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.432+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e140328c0 con 0x7f7e24109880 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.636+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f7e241a0f60 con 0x7f7e08038fc0 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.885+0000 7f7e20c00640 1 -- 172.21.15.57:0/3221729094 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f7e241a0f60 con 0x7f7e08038fc0 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.888+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 msgr2=0x7f7e0803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.888+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 0x7f7e0803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e100098c0 tx=0x7f7e10006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.888+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 msgr2=0x7f7e2419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.888+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7e140097f0 tx=0x7f7e140327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:44.599 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 shutdown_connections 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7e08038fc0 0x7f7e0803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 --2- 172.21.15.57:0/3221729094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7e24109880 0x7f7e2419d8d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 >> 172.21.15.57:0/3221729094 conn(0x7f7e2407ce60 msgr2=0x7f7e24106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 shutdown_connections 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:43.889+0000 7f7e2ac00640 1 -- 172.21.15.57:0/3221729094 wait complete. 2024-09-12T02:55:44.600 INFO:teuthology.orchestra.run.smithi057.stdout:Fetching dashboard port number... 2024-09-12T02:55:45.118 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:44 smithi057 ceph-mon[30361]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:45.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:45.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:46.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:45 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4254389947' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T02:55:46.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T02:55:46.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.117+0000 7f3e19200640 1 Processor -- start 2024-09-12T02:55:46.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.118+0000 7f3e19200640 1 -- start start 2024-09-12T02:55:46.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.118+0000 7f3e19200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.436 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.118+0000 7f3e19200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14108040 con 0x7f3e14107670 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.118+0000 7f3e11e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.119+0000 7f3e11e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45364/0 (socket says 172.21.15.57:45364) 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.119+0000 7f3e11e00640 1 -- 172.21.15.57:0/994464567 learned_addr learned my addr 172.21.15.57:0/994464567 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.119+0000 7f3e11e00640 1 -- 172.21.15.57:0/994464567 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14108870 con 0x7f3e14107670 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.120+0000 7f3e11e00640 1 --2- 172.21.15.57:0/994464567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009820 tx=0x7f3e0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf377632a78cb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.120+0000 7f3e10a00640 1 -- 172.21.15.57:0/994464567 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04037070 con 0x7f3e14107670 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.120+0000 7f3e10a00640 1 -- 172.21.15.57:0/994464567 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e04032920 con 0x7f3e14107670 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.120+0000 7f3e10a00640 1 -- 172.21.15.57:0/994464567 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04030810 con 0x7f3e14107670 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 -- 172.21.15.57:0/994464567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 msgr2=0x7f3e14107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 --2- 172.21.15.57:0/994464567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009820 tx=0x7f3e0402a270 comp rx=0 tx=0).stop 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 -- 172.21.15.57:0/994464567 shutdown_connections 2024-09-12T02:55:46.437 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 --2- 172.21.15.57:0/994464567 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e14107a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 -- 172.21.15.57:0/994464567 >> 172.21.15.57:0/994464567 conn(0x7f3e14102e00 msgr2=0x7f3e14105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 -- 172.21.15.57:0/994464567 shutdown_connections 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.121+0000 7f3e19200640 1 -- 172.21.15.57:0/994464567 wait complete. 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.122+0000 7f3e19200640 1 Processor -- start 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.122+0000 7f3e19200640 1 -- start start 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e19200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e19200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1410bc20 con 0x7f3e14107670 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e11e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e11e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45374/0 (socket says 172.21.15.57:45374) 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e11e00640 1 -- 172.21.15.57:0/4254389947 learned_addr learned my addr 172.21.15.57:0/4254389947 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.123+0000 7f3e11e00640 1 -- 172.21.15.57:0/4254389947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419de70 con 0x7f3e14107670 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e11e00640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e04030560 tx=0x7f3e04032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e04037040 con 0x7f3e14107670 2024-09-12T02:55:46.438 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e04040070 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1419e100 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e0403a740 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.124+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a0df0 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.125+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1410bda0 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.125+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3e0403b420 con 0x7f3e14107670 2024-09-12T02:55:46.439 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.126+0000 7f3e0aa00640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 0x7f3df803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.126+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e0406c6d0 con 0x7f3e14107670 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.126+0000 7f3e11400640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 0x7f3df803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.127+0000 7f3e11400640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 0x7f3df803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e000098c0 tx=0x7f3e00006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.131+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e0404a200 con 0x7f3e14107670 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.326+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3e141a0fc0 con 0x7f3e14107670 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.326+0000 7f3e0aa00640 1 -- 172.21.15.57:0/4254389947 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3e04031e80 con 0x7f3e14107670 2024-09-12T02:55:46.440 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.328+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 msgr2=0x7f3df803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.328+0000 7f3e19200640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 0x7f3df803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e000098c0 tx=0x7f3e00006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.328+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 msgr2=0x7f3e1419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.328+0000 7f3e19200640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e04030560 tx=0x7f3e04032670 comp rx=0 tx=0).stop 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 shutdown_connections 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3df8038fc0 0x7f3df803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 --2- 172.21.15.57:0/4254389947 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3e14107670 0x7f3e1419d930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 >> 172.21.15.57:0/4254389947 conn(0x7f3e14102e00 msgr2=0x7f3e14103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 shutdown_connections 2024-09-12T02:55:46.441 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:45.329+0000 7f3e19200640 1 -- 172.21.15.57:0/4254389947 wait complete. 2024-09-12T02:55:46.453 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:55:46.453 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout disabled 2024-09-12T02:55:46.466 INFO:teuthology.orchestra.run.smithi057.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:55:46.467 INFO:teuthology.orchestra.run.smithi057.stdout:systemctl: stdout inactive 2024-09-12T02:55:46.467 INFO:teuthology.orchestra.run.smithi057.stdout:firewalld.service is not enabled 2024-09-12T02:55:46.467 INFO:teuthology.orchestra.run.smithi057.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout:Ceph Dashboard is now available at: 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout: URL: https://smithi057.front.sepia.ceph.com:8443/ 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout: User: admin 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout: Password: 2xq2rn20j1 2024-09-12T02:55:46.469 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:46.470 INFO:teuthology.orchestra.run.smithi057.stdout:Saving cluster configuration to /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config directory 2024-09-12T02:55:47.005 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:46 smithi057 ceph-mon[30361]: mgrmap e13: x(active, since 6s) 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5fe00640 1 Processor -- start 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5fe00640 1 -- start start 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efc6007c850 con 0x7efc6007de20 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45382/0 (socket says 172.21.15.57:45382) 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.024+0000 7efc5ea00640 1 -- 172.21.15.57:0/2685983911 learned_addr learned my addr 172.21.15.57:0/2685983911 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.025+0000 7efc5ea00640 1 -- 172.21.15.57:0/2685983911 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6007c9d0 con 0x7efc6007de20 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.026+0000 7efc5ea00640 1 --2- 172.21.15.57:0/2685983911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc50009820 tx=0x7efc5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcb945289c01bae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:47.946 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.026+0000 7efc5e000640 1 -- 172.21.15.57:0/2685983911 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50037070 con 0x7efc6007de20 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.026+0000 7efc5e000640 1 -- 172.21.15.57:0/2685983911 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50032920 con 0x7efc6007de20 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.026+0000 7efc5e000640 1 -- 172.21.15.57:0/2685983911 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50030810 con 0x7efc6007de20 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 -- 172.21.15.57:0/2685983911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 msgr2=0x7efc6007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 --2- 172.21.15.57:0/2685983911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc50009820 tx=0x7efc5002a270 comp rx=0 tx=0).stop 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 -- 172.21.15.57:0/2685983911 shutdown_connections 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 --2- 172.21.15.57:0/2685983911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6007c280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 -- 172.21.15.57:0/2685983911 >> 172.21.15.57:0/2685983911 conn(0x7efc60102d10 msgr2=0x7efc60105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 -- 172.21.15.57:0/2685983911 shutdown_connections 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.027+0000 7efc5fe00640 1 -- 172.21.15.57:0/2685983911 wait complete. 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.028+0000 7efc5fe00640 1 Processor -- start 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.028+0000 7efc5fe00640 1 -- start start 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.028+0000 7efc5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:47.947 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010dbb0 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45390/0 (socket says 172.21.15.57:45390) 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5ea00640 1 -- 172.21.15.57:0/461740094 learned_addr learned my addr 172.21.15.57:0/461740094 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5ea00640 1 -- 172.21.15.57:0/461740094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6019dc00 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.029+0000 7efc5ea00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc500097f0 tx=0x7efc500327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.030+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc50037040 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.030+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50030cc0 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.030+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6019de90 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.030+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc5003a6b0 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.030+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc601a0b80 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.031+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efc5003baf0 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.031+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6007c300 con 0x7efc6007de20 2024-09-12T02:55:47.948 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.031+0000 7efc57400640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 0x7efc4403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.031+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efc5006c780 con 0x7efc6007de20 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.032+0000 7efc57e00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 0x7efc4403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.032+0000 7efc57e00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 0x7efc4403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc480098c0 tx=0x7efc48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.036+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc50043e10 con 0x7efc6007de20 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.314+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7efc601a0d50 con 0x7efc6007de20 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.317+0000 7efc57400640 1 -- 172.21.15.57:0/461740094 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7efc5003a850 con 0x7efc6007de20 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.319+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 msgr2=0x7efc4403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.319+0000 7efc5fe00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 0x7efc4403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc480098c0 tx=0x7efc48006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 msgr2=0x7efc6019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc500097f0 tx=0x7efc500327b0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.949 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 shutdown_connections 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7efc440390c0 0x7efc4403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 --2- 172.21.15.57:0/461740094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efc6007de20 0x7efc6019d6c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 >> 172.21.15.57:0/461740094 conn(0x7efc60102d10 msgr2=0x7efc60103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 shutdown_connections 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:/usr/bin/ceph: stderr 2024-09-12T02:55:47.320+0000 7efc5fe00640 1 -- 172.21.15.57:0/461740094 wait complete. 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.950 INFO:teuthology.orchestra.run.smithi057.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: ceph telemetry on 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout:For more information see: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:55:47.951 INFO:teuthology.orchestra.run.smithi057.stdout:Bootstrap complete. 2024-09-12T02:55:48.000 INFO:tasks.cephadm:Fetching config... 2024-09-12T02:55:48.001 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:55:48.001 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T02:55:48.038 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T02:55:48.039 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:55:48.039 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T02:55:48.116 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T02:55:48.116 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:55:48.116 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T02:55:48.211 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T02:55:48.211 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:55:48.212 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T02:55:48.286 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T02:55:48.286 DEBUG:teuthology.orchestra.run.smithi057:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:55:48.404 INFO:teuthology.orchestra.run.smithi057.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:48.429 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:55:48.467 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/461740094' entity='client.admin' 2024-09-12T02:55:48.469 INFO:teuthology.orchestra.run.smithi181.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:48.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:55:48.524 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq3+wMjKzNt6AwAwk+PvcG414H/9EuY4MW3IgBRrJLg35pxQqncCW/XQ4JDVp9zorRBj+TmXVW++4Rj05INCgLBpcfxlV5UP3CCzZKkWv94n7q+8ppj1JuvJDyw/WY7fbM7Mawaeo+D+ijw8K98aDTxYlQ5Iqecjb4Z4uZjByc8hbazcQUyzTttgDxK+kjGh6w3Aehv5SjstNaJwG+SttRQmKql5otwNBdgbh9f4O+8EmT1OS23vbHtAZAeqhthmiBWpU9lFnXw//EIQc2KgtOjX2S1G4Fl++coJ+DbWGqqW76AA2+NLkK4IxYR7mJ27qNokG5IKBld9MApKbX6FJz4eZ7c9Z7wm808+VAG805ByN1U2BsfrqMQ7Fu0m6AEl7tWmRPq2tUj6bNbTujze8NXanXKeh0ndAIIbhL1h4honuUVMic/S6K+5Jy9xd1m5koUff4NdOtcQlLPiANZYDC47HI3jolB2eaCzXaAOwhbME6QgKgy1Qc9eMhgVoTJ88= ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:55:48.537 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T02:55:48.903 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:49.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 -- 172.21.15.57:0/1455680866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 msgr2=0x7f779c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1455680866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c101800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7790009880 tx=0x7f779002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 -- 172.21.15.57:0/1455680866 shutdown_connections 2024-09-12T02:55:49.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1455680866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c101800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 -- 172.21.15.57:0/1455680866 >> 172.21.15.57:0/1455680866 conn(0x7f779c0fcb90 msgr2=0x7f779c0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 -- 172.21.15.57:0/1455680866 shutdown_connections 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.477+0000 7f77a1e00640 1 -- 172.21.15.57:0/1455680866 wait complete. 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.478+0000 7f77a1e00640 1 Processor -- start 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.478+0000 7f77a1e00640 1 -- start start 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.479+0000 7f77a1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.479+0000 7f77a1e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f779c107510 con 0x7f779c101400 2024-09-12T02:55:49.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.479+0000 7f77a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.479+0000 7f77a0a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54564/0 (socket says 172.21.15.57:54564) 2024-09-12T02:55:49.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.479+0000 7f77a0a00640 1 -- 172.21.15.57:0/1002959938 learned_addr learned my addr 172.21.15.57:0/1002959938 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:49.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f77a0a00640 1 -- 172.21.15.57:0/1002959938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c199320 con 0x7f779c101400 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f77a0a00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7790004f60 tx=0x7f77900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f779003f070 con 0x7f779c101400 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7790004cc0 con 0x7f779c101400 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7790035030 con 0x7f779c101400 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1995b0 con 0x7f779c101400 2024-09-12T02:55:49.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.480+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c19c2a0 con 0x7f779c101400 2024-09-12T02:55:49.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.482+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c107c90 con 0x7f779c101400 2024-09-12T02:55:49.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.482+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7790042070 con 0x7f779c101400 2024-09-12T02:55:49.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.482+0000 7f7799600640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 0x7f778403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.482+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f779006cb50 con 0x7f779c101400 2024-09-12T02:55:49.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.483+0000 7f779be00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 0x7f778403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.484 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.483+0000 7f779be00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 0x7f778403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77880098c0 tx=0x7f7788006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:49.488 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.487+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77900412c0 con 0x7f779c101400 2024-09-12T02:55:49.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.674+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f779c02d7c0 con 0x7f779c101400 2024-09-12T02:55:49.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.677+0000 7f7799600640 1 -- 172.21.15.57:0/1002959938 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f77900035f0 con 0x7f779c101400 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 msgr2=0x7f778403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 0x7f778403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77880098c0 tx=0x7f7788006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 msgr2=0x7f779c198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7790004f60 tx=0x7f77900048f0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 shutdown_connections 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f77840390c0 0x7f778403b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 --2- 172.21.15.57:0/1002959938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f779c101400 0x7f779c198de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 >> 172.21.15.57:0/1002959938 conn(0x7f779c0fcb90 msgr2=0x7f779c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:49.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 shutdown_connections 2024-09-12T02:55:49.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:49.680+0000 7f77a1e00640 1 -- 172.21.15.57:0/1002959938 wait complete. 2024-09-12T02:55:50.486 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T02:55:50.487 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T02:55:50.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:50 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1002959938' entity='client.admin' 2024-09-12T02:55:51.042 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.841+0000 7f4ea6000640 1 -- 172.21.15.57:0/668230199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 msgr2=0x7f4ea0076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.841+0000 7f4ea6000640 1 --2- 172.21.15.57:0/668230199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea0076fa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009880 tx=0x7f4e9002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.842+0000 7f4ea6000640 1 -- 172.21.15.57:0/668230199 shutdown_connections 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.842+0000 7f4ea6000640 1 --2- 172.21.15.57:0/668230199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea0076fa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.842+0000 7f4ea6000640 1 -- 172.21.15.57:0/668230199 >> 172.21.15.57:0/668230199 conn(0x7f4ea00fefc0 msgr2=0x7f4ea0101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.842+0000 7f4ea6000640 1 -- 172.21.15.57:0/668230199 shutdown_connections 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.842+0000 7f4ea6000640 1 -- 172.21.15.57:0/668230199 wait complete. 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.843+0000 7f4ea6000640 1 Processor -- start 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.843+0000 7f4ea6000640 1 -- start start 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4ea6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:52.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4ea6000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea010dcf0 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4e9f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4e9f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54580/0 (socket says 172.21.15.57:54580) 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4e9f400640 1 -- 172.21.15.57:0/2657235410 learned_addr learned my addr 172.21.15.57:0/2657235410 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.844+0000 7f4e9f400640 1 -- 172.21.15.57:0/2657235410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea019d860 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4e9f400640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e90004f60 tx=0x7f4e900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e9003f070 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e90003ad0 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e90035030 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea019daf0 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.845+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea01a07e0 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.846+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4e90042070 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.847+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea010e470 con 0x7f4ea0076ba0 2024-09-12T02:55:52.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.847+0000 7f4e97e00640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 0x7f4e8403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.847+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e9006ce40 con 0x7f4ea0076ba0 2024-09-12T02:55:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.847+0000 7f4e9ea00640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 0x7f4e8403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.848+0000 7f4e9ea00640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 0x7f4e8403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098c0 tx=0x7f4e8c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:52.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:51.853+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e9004c350 con 0x7f4ea0076ba0 2024-09-12T02:55:52.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.046+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4ea0062630 con 0x7f4e840390c0 2024-09-12T02:55:52.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.051+0000 7f4e97e00640 1 -- 172.21.15.57:0/2657235410 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ea0062630 con 0x7f4e840390c0 2024-09-12T02:55:52.053 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 msgr2=0x7f4e8403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 0x7f4e8403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0098c0 tx=0x7f4e8c006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 msgr2=0x7f4ea019d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4e90004f60 tx=0x7f4e900048f0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 shutdown_connections 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4e840390c0 0x7f4e8403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 --2- 172.21.15.57:0/2657235410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4ea0076ba0 0x7f4ea019d320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 >> 172.21.15.57:0/2657235410 conn(0x7f4ea00fefc0 msgr2=0x7f4ea00ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 shutdown_connections 2024-09-12T02:55:52.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:52.053+0000 7f4ea6000640 1 -- 172.21.15.57:0/2657235410 wait complete. 2024-09-12T02:55:53.032 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi181 2024-09-12T02:55:53.033 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:55:53.033 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:55:53.053 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:55:53.054 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:55:53.111 INFO:tasks.cephadm:Adding host smithi181 to orchestrator... 2024-09-12T02:55:53.112 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch host add smithi181 2024-09-12T02:55:53.321 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:53 smithi057 ceph-mon[30361]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:53.321 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:53 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:53.821 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 -- 172.21.15.57:0/3361939977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 msgr2=0x7f2f44074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 --2- 172.21.15.57:0/3361939977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f44074680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009880 tx=0x7f2f3402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 -- 172.21.15.57:0/3361939977 shutdown_connections 2024-09-12T02:55:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 --2- 172.21.15.57:0/3361939977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f44074680 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:54.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 -- 172.21.15.57:0/3361939977 >> 172.21.15.57:0/3361939977 conn(0x7f2f440fcb90 msgr2=0x7f2f440fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 -- 172.21.15.57:0/3361939977 shutdown_connections 2024-09-12T02:55:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.391+0000 7f2f49000640 1 -- 172.21.15.57:0/3361939977 wait complete. 2024-09-12T02:55:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.393+0000 7f2f49000640 1 Processor -- start 2024-09-12T02:55:54.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.393+0000 7f2f49000640 1 -- start start 2024-09-12T02:55:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.393+0000 7f2f49000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.393+0000 7f2f49000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4410db60 con 0x7f2f44076220 2024-09-12T02:55:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.394+0000 7f2f43400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.394+0000 7f2f43400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54600/0 (socket says 172.21.15.57:54600) 2024-09-12T02:55:54.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.394+0000 7f2f43400640 1 -- 172.21.15.57:0/2169467346 learned_addr learned my addr 172.21.15.57:0/2169467346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:55:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.394+0000 7f2f43400640 1 -- 172.21.15.57:0/2169467346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419d710 con 0x7f2f44076220 2024-09-12T02:55:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.394+0000 7f2f43400640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004f60 tx=0x7f2f340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.395+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f3403f070 con 0x7f2f44076220 2024-09-12T02:55:54.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.395+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f34004cc0 con 0x7f2f44076220 2024-09-12T02:55:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.395+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4419d9a0 con 0x7f2f44076220 2024-09-12T02:55:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.395+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a0690 con 0x7f2f44076220 2024-09-12T02:55:54.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.395+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f34035030 con 0x7f2f44076220 2024-09-12T02:55:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.396+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f240052c0 con 0x7f2f44076220 2024-09-12T02:55:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.396+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2f34042070 con 0x7f2f44076220 2024-09-12T02:55:54.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.397+0000 7f2f3be00640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 0x7f2f2803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.397+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f3406cdb0 con 0x7f2f44076220 2024-09-12T02:55:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.397+0000 7f2f42a00640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 0x7f2f2803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:54.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.398+0000 7f2f42a00640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 0x7f2f2803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2f300098c0 tx=0x7f2f30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:54.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.401+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f340433b0 con 0x7f2f44076220 2024-09-12T02:55:54.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:54.609+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}) -- 0x7f2f24000bf0 con 0x7f2f280390c0 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi057", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: Updating smithi057:/etc/ceph/ceph.conf 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:55:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:54 smithi057 ceph-mon[30361]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi181", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:58.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:57 smithi057 ceph-mon[30361]: Deploying cephadm binary to smithi181 2024-09-12T02:55:58.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.520+0000 7f2f3be00640 1 -- 172.21.15.57:0/2169467346 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2f24000bf0 con 0x7f2f280390c0 2024-09-12T02:55:58.520 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi181' with addr '172.21.15.181' 2024-09-12T02:55:58.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 msgr2=0x7f2f2803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:58.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 0x7f2f2803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2f300098c0 tx=0x7f2f30006db0 comp rx=0 tx=0).stop 2024-09-12T02:55:58.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 msgr2=0x7f2f4419d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:58.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004f60 tx=0x7f2f340048f0 comp rx=0 tx=0).stop 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 shutdown_connections 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2f280390c0 0x7f2f2803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 --2- 172.21.15.57:0/2169467346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2f44076220 0x7f2f4419d1d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.522+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 >> 172.21.15.57:0/2169467346 conn(0x7f2f440fcb90 msgr2=0x7f2f440fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.523+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 shutdown_connections 2024-09-12T02:55:58.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:55:58.523+0000 7f2f49000640 1 -- 172.21.15.57:0/2169467346 wait complete. 2024-09-12T02:55:59.113 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:55:59.478 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:59.775 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:59.776 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:59 smithi057 ceph-mon[30361]: Added host smithi181 2024-09-12T02:55:59.776 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:59.776 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:55:59.776 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:55:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:00.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 -- 172.21.15.57:0/554097700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 msgr2=0x7f81e0109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 --2- 172.21.15.57:0/554097700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0109b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009880 tx=0x7f81d002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 -- 172.21.15.57:0/554097700 shutdown_connections 2024-09-12T02:56:00.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 --2- 172.21.15.57:0/554097700 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0109b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 -- 172.21.15.57:0/554097700 >> 172.21.15.57:0/554097700 conn(0x7f81e00ff480 msgr2=0x7f81e01018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:00.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 -- 172.21.15.57:0/554097700 shutdown_connections 2024-09-12T02:56:00.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.080+0000 7f81e5c00640 1 -- 172.21.15.57:0/554097700 wait complete. 2024-09-12T02:56:00.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.081+0000 7f81e5c00640 1 Processor -- start 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.081+0000 7f81e5c00640 1 -- start start 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81e5c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81e5c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f81e010de40 con 0x7f81e0109780 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81dea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81dea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56828/0 (socket says 172.21.15.57:56828) 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81dea00640 1 -- 172.21.15.57:0/1341782624 learned_addr learned my addr 172.21.15.57:0/1341782624 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:00.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.082+0000 7f81dea00640 1 -- 172.21.15.57:0/1341782624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e0199860 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81dea00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009230 tx=0x7f81d0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d003f070 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d003bd40 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d0035030 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e0199b50 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.083+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e019dbf0 con 0x7f81e0109780 2024-09-12T02:56:00.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.084+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e0104ec0 con 0x7f81e0109780 2024-09-12T02:56:00.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.084+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f81d0042070 con 0x7f81e0109780 2024-09-12T02:56:00.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.084+0000 7f81d7400640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 0x7f81c403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.085+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f81d006ccc0 con 0x7f81e0109780 2024-09-12T02:56:00.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.085+0000 7f81d7e00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 0x7f81c403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.085+0000 7f81d7e00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 0x7f81c403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81c80098c0 tx=0x7f81c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:00.089 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.090+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81d0073080 con 0x7f81e0109780 2024-09-12T02:56:00.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.282+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81e0109be0 con 0x7f81c4039060 2024-09-12T02:56:00.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.283+0000 7f81d7400640 1 -- 172.21.15.57:0/1341782624 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f81e0109be0 con 0x7f81c4039060 2024-09-12T02:56:00.283 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:56:00.283 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}] 2024-09-12T02:56:00.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 msgr2=0x7f81c403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 0x7f81c403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81c80098c0 tx=0x7f81c8006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 msgr2=0x7f81e0199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009230 tx=0x7f81d0003d80 comp rx=0 tx=0).stop 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 shutdown_connections 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f81c4039060 0x7f81c403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 --2- 172.21.15.57:0/1341782624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f81e0109780 0x7f81e0199320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 >> 172.21.15.57:0/1341782624 conn(0x7f81e00ff480 msgr2=0x7f81e0100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 shutdown_connections 2024-09-12T02:56:00.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:00.286+0000 7f81e5c00640 1 -- 172.21.15.57:0/1341782624 wait complete. 2024-09-12T02:56:00.969 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:00 smithi057 ceph-mon[30361]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:00.970 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:00.970 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:00 smithi057 ceph-mon[30361]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:01.006 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-09-12T02:56:01.006 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:56:01.006 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:56:01.026 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:56:01.026 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:01.084 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-09-12T02:56:01.084 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch host add smithi204 2024-09-12T02:56:01.447 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 -- 172.21.15.57:0/232386296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 msgr2=0x7f8604074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 --2- 172.21.15.57:0/232386296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f8604074a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009880 tx=0x7f85f802a670 comp rx=0 tx=0).stop 2024-09-12T02:56:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 -- 172.21.15.57:0/232386296 shutdown_connections 2024-09-12T02:56:01.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 --2- 172.21.15.57:0/232386296 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f8604074a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 -- 172.21.15.57:0/232386296 >> 172.21.15.57:0/232386296 conn(0x7f86040fd290 msgr2=0x7f86040ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 -- 172.21.15.57:0/232386296 shutdown_connections 2024-09-12T02:56:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.919+0000 7f860c800640 1 -- 172.21.15.57:0/232386296 wait complete. 2024-09-12T02:56:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.920+0000 7f860c800640 1 Processor -- start 2024-09-12T02:56:01.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f860c800640 1 -- start start 2024-09-12T02:56:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f860c800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f860c800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f860410dec0 con 0x7f8604074620 2024-09-12T02:56:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f8609e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f8609e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56848/0 (socket says 172.21.15.57:56848) 2024-09-12T02:56:01.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.921+0000 7f8609e00640 1 -- 172.21.15.57:0/3808676308 learned_addr learned my addr 172.21.15.57:0/3808676308 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f8609e00640 1 -- 172.21.15.57:0/3808676308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8604199920 con 0x7f8604074620 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f8609e00640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009230 tx=0x7f85f8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f803f070 con 0x7f8604074620 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f803bd40 con 0x7f8604074620 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f8035030 con 0x7f8604074620 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8604199bb0 con 0x7f8604074620 2024-09-12T02:56:01.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.922+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860419a010 con 0x7f8604074620 2024-09-12T02:56:01.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.923+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f85f8042070 con 0x7f8604074620 2024-09-12T02:56:01.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.923+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8604075cd0 con 0x7f8604074620 2024-09-12T02:56:01.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.924+0000 7f85fea00640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 0x7f85ec03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:01.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.924+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f85f806ccb0 con 0x7f8604074620 2024-09-12T02:56:01.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.924+0000 7f8609400640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 0x7f85ec03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:01.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.925+0000 7f8609400640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 0x7f85ec03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85f40098c0 tx=0x7f85f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:01.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:01.929+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85f80412c0 con 0x7f8604074620 2024-09-12T02:56:02.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:02.117+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7f860419a7b0 con 0x7f85ec0390c0 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi181", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:02 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: Deploying cephadm binary to smithi204 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:04 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:05.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:05 smithi057 ceph-mon[30361]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:06.074 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.073+0000 7f85fea00640 1 -- 172.21.15.57:0/3808676308 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f860419a7b0 con 0x7f85ec0390c0 2024-09-12T02:56:06.074 INFO:teuthology.orchestra.run.smithi057.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 msgr2=0x7f85ec03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 0x7f85ec03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85f40098c0 tx=0x7f85f4006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 msgr2=0x7f86041993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009230 tx=0x7f85f8003d80 comp rx=0 tx=0).stop 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 shutdown_connections 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85ec0390c0 0x7f85ec03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 --2- 172.21.15.57:0/3808676308 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8604074620 0x7f86041993e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:06.077 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 >> 172.21.15.57:0/3808676308 conn(0x7f86040fd290 msgr2=0x7f8604107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:06.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 shutdown_connections 2024-09-12T02:56:06.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:06.076+0000 7f860c800640 1 -- 172.21.15.57:0/3808676308 wait complete. 2024-09-12T02:56:06.736 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:56:07.101 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:07.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:07.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:07.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: Added host smithi204 2024-09-12T02:56:07.354 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:07.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:07.355 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.623+0000 7f1291400640 1 -- 172.21.15.57:0/1020272201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 msgr2=0x7f128c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.628 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 --2- 172.21.15.57:0/1020272201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c109b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f127c009880 tx=0x7f127c02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 -- 172.21.15.57:0/1020272201 shutdown_connections 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 --2- 172.21.15.57:0/1020272201 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c109b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 -- 172.21.15.57:0/1020272201 >> 172.21.15.57:0/1020272201 conn(0x7f128c0ff2c0 msgr2=0x7f128c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 -- 172.21.15.57:0/1020272201 shutdown_connections 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.624+0000 7f1291400640 1 -- 172.21.15.57:0/1020272201 wait complete. 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.625+0000 7f1291400640 1 Processor -- start 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.625+0000 7f1291400640 1 -- start start 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f1291400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f1291400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f128c10dd50 con 0x7f128c109740 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f128be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f128be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56866/0 (socket says 172.21.15.57:56866) 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f128be00640 1 -- 172.21.15.57:0/1400223896 learned_addr learned my addr 172.21.15.57:0/1400223896 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:07.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f128be00640 1 -- 172.21.15.57:0/1400223896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c199720 con 0x7f128c109740 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.626+0000 7f128be00640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f127c009230 tx=0x7f127c003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.627+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f127c03f070 con 0x7f128c109740 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.627+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f127c03bd40 con 0x7f128c109740 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.627+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c199a10 con 0x7f128c109740 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.627+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f127c035030 con 0x7f128c109740 2024-09-12T02:56:07.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.627+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c19dab0 con 0x7f128c109740 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.628+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c104ce0 con 0x7f128c109740 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.628+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f127c042070 con 0x7f128c109740 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.629+0000 7f1288c00640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 0x7f127003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.629+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f127c06c890 con 0x7f128c109740 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.629+0000 7f128b400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 0x7f127003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:07.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.630+0000 7f128b400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 0x7f127003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12780098c0 tx=0x7f1278006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:07.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.634+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f127c040260 con 0x7f128c109740 2024-09-12T02:56:07.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.822+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f128c109bd0 con 0x7f12700390c0 2024-09-12T02:56:07.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.823+0000 7f1288c00640 1 -- 172.21.15.57:0/1400223896 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f128c109bd0 con 0x7f12700390c0 2024-09-12T02:56:07.824 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:56:07.824 INFO:teuthology.orchestra.run.smithi057.stdout:[{"addr": "172.21.15.57", "hostname": "smithi057", "labels": [], "status": ""}, {"addr": "172.21.15.181", "hostname": "smithi181", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-09-12T02:56:07.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 msgr2=0x7f127003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 0x7f127003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f12780098c0 tx=0x7f1278006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 msgr2=0x7f128c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f127c009230 tx=0x7f127c003d80 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 shutdown_connections 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f12700390c0 0x7f127003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 --2- 172.21.15.57:0/1400223896 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f128c109740 0x7f128c1991e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.825+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 >> 172.21.15.57:0/1400223896 conn(0x7f128c0ff2c0 msgr2=0x7f128c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.826+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 shutdown_connections 2024-09-12T02:56:07.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:07.826+0000 7f1291400640 1 -- 172.21.15.57:0/1400223896 wait complete. 2024-09-12T02:56:08.474 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:08.511 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T02:56:08.511 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T02:56:08.872 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 -- 172.21.15.57:0/1219875074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 msgr2=0x7f7718109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 --2- 172.21.15.57:0/1219875074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f7718109b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7708009880 tx=0x7f770802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 -- 172.21.15.57:0/1219875074 shutdown_connections 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 --2- 172.21.15.57:0/1219875074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f7718109b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 -- 172.21.15.57:0/1219875074 >> 172.21.15.57:0/1219875074 conn(0x7f77180ff450 msgr2=0x7f7718101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 -- 172.21.15.57:0/1219875074 shutdown_connections 2024-09-12T02:56:09.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.348+0000 7f771ce00640 1 -- 172.21.15.57:0/1219875074 wait complete. 2024-09-12T02:56:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.349+0000 7f771ce00640 1 Processor -- start 2024-09-12T02:56:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.349+0000 7f771ce00640 1 -- start start 2024-09-12T02:56:09.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f771ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f771ce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f771810de30 con 0x7f7718109770 2024-09-12T02:56:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f7715e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f7715e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54108/0 (socket says 172.21.15.57:54108) 2024-09-12T02:56:09.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f7715e00640 1 -- 172.21.15.57:0/3900365795 learned_addr learned my addr 172.21.15.57:0/3900365795 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.350+0000 7f7715e00640 1 -- 172.21.15.57:0/3900365795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181998e0 con 0x7f7718109770 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f7715e00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7708009230 tx=0x7f7708003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f770803f070 con 0x7f7718109770 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770803bc80 con 0x7f7718109770 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7708035030 con 0x7f7718109770 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7718199bd0 con 0x7f7718109770 2024-09-12T02:56:09.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.351+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771819dc70 con 0x7f7718109770 2024-09-12T02:56:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.352+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7708042070 con 0x7f7718109770 2024-09-12T02:56:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.352+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718104eb0 con 0x7f7718109770 2024-09-12T02:56:09.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.353+0000 7f770ea00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 0x7f76fc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:09.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.353+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f770806cd50 con 0x7f7718109770 2024-09-12T02:56:09.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.353+0000 7f7715400640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 0x7f76fc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:09.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.354+0000 7f7715400640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 0x7f76fc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f77040098c0 tx=0x7f7704006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:09.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.358+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7708031e20 con 0x7f7718109770 2024-09-12T02:56:09.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:09.548+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f771819a700 con 0x7f7718109770 2024-09-12T02:56:09.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:09 smithi057 ceph-mon[30361]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:09.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:09 smithi057 ceph-mon[30361]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:10.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.431+0000 7f770ea00640 1 -- 172.21.15.57:0/3900365795 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f77080372d0 con 0x7f7718109770 2024-09-12T02:56:10.432 INFO:teuthology.orchestra.run.smithi057.stderr:adjusted tunables profile to default 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 msgr2=0x7f76fc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 0x7f76fc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f77040098c0 tx=0x7f7704006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 msgr2=0x7f77181993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7708009230 tx=0x7f7708003d40 comp rx=0 tx=0).stop 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 shutdown_connections 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f76fc039060 0x7f76fc03b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 --2- 172.21.15.57:0/3900365795 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7718109770 0x7f77181993a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 >> 172.21.15.57:0/3900365795 conn(0x7f77180ff450 msgr2=0x7f7718107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:10.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 shutdown_connections 2024-09-12T02:56:10.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:10.434+0000 7f771ce00640 1 -- 172.21.15.57:0/3900365795 wait complete. 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3900365795' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:11.132 INFO:tasks.cephadm:Adding mon.a on smithi057 2024-09-12T02:56:11.132 INFO:tasks.cephadm:Adding mon.b on smithi181 2024-09-12T02:56:11.132 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-09-12T02:56:11.132 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c' 2024-09-12T02:56:11.376 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 -- 172.21.15.204:0/2367893238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 msgr2=0x7f98ac106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 --2- 172.21.15.204:0/2367893238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac106940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009880 tx=0x7f98a802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 -- 172.21.15.204:0/2367893238 shutdown_connections 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 --2- 172.21.15.204:0/2367893238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac106940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 -- 172.21.15.204:0/2367893238 >> 172.21.15.204:0/2367893238 conn(0x7f98ac0fc1b0 msgr2=0x7f98ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 -- 172.21.15.204:0/2367893238 shutdown_connections 2024-09-12T02:56:11.800 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.799+0000 7f98b42f7640 1 -- 172.21.15.204:0/2367893238 wait complete. 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.800+0000 7f98b42f7640 1 Processor -- start 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.800+0000 7f98b42f7640 1 -- start start 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.800+0000 7f98b42f7640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.800+0000 7f98b42f7640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac10ab90 con 0x7f98ac106560 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b2093640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b2093640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.204:39822/0 (socket says 172.21.15.204:39822) 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b2093640 1 -- 172.21.15.204:0/1783875255 learned_addr learned my addr 172.21.15.204:0/1783875255 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b2093640 1 -- 172.21.15.204:0/1783875255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac198490 con 0x7f98ac106560 2024-09-12T02:56:11.802 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b2093640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009230 tx=0x7f98a8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:11.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a803f070 con 0x7f98ac106560 2024-09-12T02:56:11.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac198720 con 0x7f98ac106560 2024-09-12T02:56:11.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a803bc80 con 0x7f98ac106560 2024-09-12T02:56:11.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98a8035030 con 0x7f98ac106560 2024-09-12T02:56:11.803 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.801+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac198b80 con 0x7f98ac106560 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.802+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f98a8042070 con 0x7f98ac106560 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.802+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ac101bd0 con 0x7f98ac106560 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.803+0000 7f98a37fe640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 0x7f989803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.803+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98a806c9e0 con 0x7f98ac106560 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.803+0000 7f98b1892640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 0x7f989803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:11.804 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.803+0000 7f98b1892640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 0x7f989803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f989c0098c0 tx=0x7f989c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:11.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.808+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98a8031570 con 0x7f98ac106560 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3900365795' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:11.985 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.985+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7f98ac06a4c0 con 0x7f9898039060 2024-09-12T02:56:11.992 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.992+0000 7f98a37fe640 1 -- 172.21.15.204:0/1783875255 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f98ac06a4c0 con 0x7f9898039060 2024-09-12T02:56:11.992 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 msgr2=0x7f989803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 0x7f989803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f989c0098c0 tx=0x7f989c006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 msgr2=0x7f98ac197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009230 tx=0x7f98a8003d40 comp rx=0 tx=0).stop 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 shutdown_connections 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9898039060 0x7f989803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 --2- 172.21.15.204:0/1783875255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f98ac106560 0x7f98ac197f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 >> 172.21.15.204:0/1783875255 conn(0x7f98ac0fc1b0 msgr2=0x7f98ac0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 shutdown_connections 2024-09-12T02:56:11.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:11.994+0000 7f98b42f7640 1 -- 172.21.15.204:0/1783875255 wait complete. 2024-09-12T02:56:12.638 DEBUG:teuthology.orchestra.run.smithi181:mon.b> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T02:56:12.640 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T02:56:12.643 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:56:12.643 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:13.587 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:13 smithi057 ceph-mon[30361]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:13 smithi057 ceph-mon[30361]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:13 smithi057 ceph-mon[30361]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-09-12T02:56:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:13 smithi057 ceph-mon[30361]: Deploying daemon mon.c on smithi204 2024-09-12T02:56:14.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:14 smithi057 ceph-mon[30361]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:15.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 -- 172.21.15.204:0/690386843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 msgr2=0x7f2388106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:15.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 --2- 172.21.15.204:0/690386843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2378009810 tx=0x7f237802a540 comp rx=0 tx=0).stop 2024-09-12T02:56:15.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 -- 172.21.15.204:0/690386843 shutdown_connections 2024-09-12T02:56:15.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 --2- 172.21.15.204:0/690386843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:15.023 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 -- 172.21.15.204:0/690386843 >> 172.21.15.204:0/690386843 conn(0x7f2388077310 msgr2=0x7f23880fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 -- 172.21.15.204:0/690386843 shutdown_connections 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.978+0000 7f238d26c640 1 -- 172.21.15.204:0/690386843 wait complete. 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.979+0000 7f238d26c640 1 Processor -- start 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.979+0000 7f238d26c640 1 -- start start 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.979+0000 7f238d26c640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.979+0000 7f238d26c640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f238810ab70 con 0x7f2388106540 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.980+0000 7f2386ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.980+0000 7f2386ffd640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.204:39832/0 (socket says 172.21.15.204:39832) 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.980+0000 7f2386ffd640 1 -- 172.21.15.204:0/2456246534 learned_addr learned my addr 172.21.15.204:0/2456246534 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.980+0000 7f2386ffd640 1 -- 172.21.15.204:0/2456246534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238806a4b0 con 0x7f2388106540 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.981+0000 7f2386ffd640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23780045e0 tx=0x7f2378004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.981+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2378030e70 con 0x7f2388106540 2024-09-12T02:56:15.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.981+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23780318e0 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.981+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2388196300 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.982+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881966a0 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.982+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23780049c0 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.986+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2378039040 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.986+0000 7f2384ff9640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 0x7f237403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.987+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f237806d1d0 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.988+0000 7f237e7fc640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 0x7f237403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.989+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2368005290 con 0x7f2388106540 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.990+0000 7f237e7fc640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 0x7f237403b620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f2370006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:15.025 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:14.995+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2378040030 con 0x7f2388106540 2024-09-12T02:56:15.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.247+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2368005580 con 0x7f2388106540 2024-09-12T02:56:15.248 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.247+0000 7f2384ff9640 1 -- 172.21.15.204:0/2456246534 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f237803ae40 con 0x7f2388106540 2024-09-12T02:56:15.248 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:56:15.248 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","modified":"2024-09-12T02:54:40.839595Z","created":"2024-09-12T02:54:40.839595Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:56:15.248 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 msgr2=0x7f237403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 0x7f237403b620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f2370006db0 comp rx=0 tx=0).stop 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 msgr2=0x7f2388195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f23780045e0 tx=0x7f2378004610 comp rx=0 tx=0).stop 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 shutdown_connections 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2374039160 0x7f237403b620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 --2- 172.21.15.204:0/2456246534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2388106540 0x7f2388195dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 >> 172.21.15.204:0/2456246534 conn(0x7f2388077310 msgr2=0x7f23880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 shutdown_connections 2024-09-12T02:56:15.251 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:15.250+0000 7f238d26c640 1 -- 172.21.15.204:0/2456246534 wait complete. 2024-09-12T02:56:15.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:15 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/2456246534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:15.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:15 smithi204 systemd[1]: Starting Ceph mon.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:56:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 podman[34994]: 2024-09-12 02:56:16.169099572 +0000 UTC m=+0.108682058 container create 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T02:56:16.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 podman[34994]: 2024-09-12 02:56:16.076265001 +0000 UTC m=+0.015847486 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:56:16.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 podman[34994]: 2024-09-12 02:56:16.363166287 +0000 UTC m=+0.302748776 container init 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T02:56:16.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 podman[34994]: 2024-09-12 02:56:16.366232374 +0000 UTC m=+0.305814851 container start 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pidfile_write: ignore empty --pid-file 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: load: jerasure load: lrc 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Git sha 0 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: DB SUMMARY 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: DB Session ID: 6B9WNK2AQBKI5ZLYXJ0F 2024-09-12T02:56:16.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:56:16.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.env: 0x564a7c75a940 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.info_log: 0x564a7eb9f6e0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.statistics: (nil) 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.use_fsync: 0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:56:16.647 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.db_log_dir: 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.wal_dir: 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:56:16.648 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.write_buffer_manager: 0x564a7ebaedc0 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:56:16.649 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.unordered_write: 0 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:56:16.650 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:56:16.651 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.row_cache: None 2024-09-12T02:56:16.651 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.wal_filter: None 2024-09-12T02:56:16.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.wal_compression: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:56:16.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:56:16.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_open_files: -1 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:56:16.655 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Compression algorithms supported: 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kZSTD supported: 0 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:56:16.656 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.merge_operator: 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_filter: None 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564a7eb9f320) 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-09-12T02:56:16.657 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x564a7eb931f0 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-09-12T02:56:16.658 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-09-12T02:56:16.659 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression: NoCompression 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.num_levels: 7 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:56:16.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:56:16.661 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:56:16.662 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:56:16.663 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:56:16.664 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.ttl: 2592000 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:56:16.665 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cf480ae6-4b7f-4765-8b53-55d872698692 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109776393360, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109776393752, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109776, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cf480ae6-4b7f-4765-8b53-55d872698692", "db_session_id": "6B9WNK2AQBKI5ZLYXJ0F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:56:16.666 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109776393844, "job": 1, "event": "recovery_finished"} 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564a7ebc0e00 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: DB pointer 0x564a7ecbc000 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:56:16.667 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:56:16.668 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x564a7eb931f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T02:56:16.669 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(???) e0 preinit fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: btime 2024-09-12T02:54:45:449413+0000 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-09-12T02:56:16.670 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.671 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Added host smithi204 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3900365795' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.672 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3900365795' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:16.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Saving service mon spec with placement smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: Deploying daemon mon.c on smithi204 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/2456246534' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 ceph-mon[35013]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 bash[34994]: 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 2024-09-12T02:56:16.674 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:16 smithi204 systemd[1]: Started Ceph mon.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:56:17.010 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:56:17.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:56:17.265 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:56:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: mon.a calling monitor election 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: mon.c calling monitor election 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: monmap epoch 2 2024-09-12T02:56:21.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: last_changed 2024-09-12T02:56:16.422169+0000 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: min_mon_release 19 (squid) 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: election_strategy: 1 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: fsmap 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: mgrmap e13: x(active, since 41s) 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: overall HEALTH_OK 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:21.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: mon.a calling monitor election 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: mon.c calling monitor election 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: monmap epoch 2 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: last_changed 2024-09-12T02:56:16.422169+0000 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: min_mon_release 19 (squid) 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: election_strategy: 1 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:21.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: fsmap 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: mgrmap e13: x(active, since 41s) 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: overall HEALTH_OK 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:21.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:22.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 -- 172.21.15.204:0/3934060713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 msgr2=0x7ff4180059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 --2- 172.21.15.204:0/3934060713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4180059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4301016a0 tx=0x7ff42002c490 comp rx=0 tx=0).stop 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 -- 172.21.15.204:0/3934060713 shutdown_connections 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 --2- 172.21.15.204:0/3934060713 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4180059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 -- 172.21.15.204:0/3934060713 >> 172.21.15.204:0/3934060713 conn(0x7ff4300fc220 msgr2=0x7ff4300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 -- 172.21.15.204:0/3934060713 shutdown_connections 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.134+0000 7ff436866640 1 -- 172.21.15.204:0/3934060713 wait complete. 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.135+0000 7ff436866640 1 Processor -- start 2024-09-12T02:56:22.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.135+0000 7ff436866640 1 -- start start 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff436866640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4301980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff436866640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff436866640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff430108670 con 0x7ff418003530 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff436866640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301084f0 con 0x7ff430198600 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52670/0 (socket says 172.21.15.204:52670) 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 -- 172.21.15.204:0/1151791198 learned_addr learned my addr 172.21.15.204:0/1151791198 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42ffff640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4301980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 -- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 msgr2=0x7ff430198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 -- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 msgr2=0x7ff430198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:56:22.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42f7fe640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42ffff640 1 -- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 msgr2=0x7ff430198a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42ffff640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.136+0000 7ff42ffff640 1 -- 172.21.15.204:0/1151791198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4301a1130 con 0x7ff418003530 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff42ffff640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4301980c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff42002ca80 tx=0x7ff4200339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff4200329e0 con 0x7ff418003530 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff420005520 con 0x7ff418003530 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4301a1300 con 0x7ff418003530 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff420004bc0 con 0x7ff418003530 2024-09-12T02:56:22.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.137+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301a1800 con 0x7ff418003530 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.138+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff420043020 con 0x7ff418003530 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.138+0000 7ff42d7fa640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 0x7ff41c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.138+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff430101ca0 con 0x7ff418003530 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.138+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff420041020 con 0x7ff418003530 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.139+0000 7ff42f7fe640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 0x7ff41c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:22.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.139+0000 7ff42f7fe640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 0x7ff41c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff424009100 tx=0x7ff424008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:22.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.144+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff42004a020 con 0x7ff418003530 2024-09-12T02:56:22.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.379+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff43002d7c0 con 0x7ff418003530 2024-09-12T02:56:22.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.379+0000 7ff42d7fa640 1 -- 172.21.15.204:0/1151791198 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff420030260 con 0x7ff418003530 2024-09-12T02:56:22.380 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:56:22.381 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","modified":"2024-09-12T02:56:16.422169Z","created":"2024-09-12T02:54:40.839595Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:56:22.381 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-12T02:56:22.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.381+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 msgr2=0x7ff41c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:22.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.381+0000 7ff436866640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 0x7ff41c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff424009100 tx=0x7ff424008040 comp rx=0 tx=0).stop 2024-09-12T02:56:22.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.381+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 msgr2=0x7ff4301980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:22.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.381+0000 7ff436866640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4301980c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff42002ca80 tx=0x7ff4200339a0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 shutdown_connections 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff41c038e00 0x7ff41c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff430198600 0x7ff430198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 --2- 172.21.15.204:0/1151791198 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff418003530 0x7ff4301980c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 >> 172.21.15.204:0/1151791198 conn(0x7ff4300fc220 msgr2=0x7ff4300fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 shutdown_connections 2024-09-12T02:56:22.383 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:22.382+0000 7ff436866640 1 -- 172.21.15.204:0/1151791198 wait complete. 2024-09-12T02:56:22.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:22 smithi204 ceph-mon[35013]: Deploying daemon mon.b on smithi181 2024-09-12T02:56:22.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:22 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/1151791198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:22 smithi057 ceph-mon[30361]: Deploying daemon mon.b on smithi181 2024-09-12T02:56:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:22 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/1151791198' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:23 smithi057 ceph-mon[30361]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:23 smithi204 ceph-mon[35013]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:23.962 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:56:23.962 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:56:24.216 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 -- 172.21.15.204:0/1668573352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c076720 msgr2=0x7fd02c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 --2- 172.21.15.204:0/1668573352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c076720 0x7fd02c076b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd028009880 tx=0x7fd02802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 -- 172.21.15.204:0/1668573352 shutdown_connections 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 --2- 172.21.15.204:0/1668573352 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c076720 0x7fd02c076b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 -- 172.21.15.204:0/1668573352 >> 172.21.15.204:0/1668573352 conn(0x7fd02c0ff510 msgr2=0x7fd02c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:24.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 -- 172.21.15.204:0/1668573352 shutdown_connections 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.627+0000 7fd033622640 1 -- 172.21.15.204:0/1668573352 wait complete. 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 Processor -- start 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 -- start start 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 0x7fd02c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c112380 con 0x7fd02c1a3c00 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd033622640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c112200 con 0x7fd02c076720 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52684/0 (socket says 172.21.15.204:52684) 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 -- 172.21.15.204:0/3624852261 learned_addr learned my addr 172.21.15.204:0/3624852261 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 -- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 msgr2=0x7fd02c1a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 -- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 msgr2=0x7fd02c1a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:56:24.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.628+0000 7fd0313be640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd0313be640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd030bbd640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 0x7fd02c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd030bbd640 1 -- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 msgr2=0x7fd02c1a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd030bbd640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd030bbd640 1 -- 172.21.15.204:0/3624852261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c1a4210 con 0x7fd02c1a3c00 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd030bbd640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 0x7fd02c19d840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b3d0 tx=0x7fd01c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd01c011630 con 0x7fd02c1a3c00 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c19dd80 con 0x7fd02c1a3c00 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c011cb0 con 0x7fd02c1a3c00 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd01c010df0 con 0x7fd02c1a3c00 2024-09-12T02:56:24.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.629+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c19e310 con 0x7fd02c1a3c00 2024-09-12T02:56:24.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.630+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02c077e60 con 0x7fd02c1a3c00 2024-09-12T02:56:24.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.630+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd01c018020 con 0x7fd02c1a3c00 2024-09-12T02:56:24.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.631+0000 7fd0227fc640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 0x7fd01803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:24.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.631+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd01c04bc80 con 0x7fd02c1a3c00 2024-09-12T02:56:24.633 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.631+0000 7fd0313be640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 0x7fd01803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:24.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.632+0000 7fd0313be640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 0x7fd01803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0280096c0 tx=0x7fd028031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:24.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.636+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd01c01f020 con 0x7fd02c1a3c00 2024-09-12T02:56:24.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.866+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd02c19e620 con 0x7fd02c1a3c00 2024-09-12T02:56:24.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.866+0000 7fd0227fc640 1 -- 172.21.15.204:0/3624852261 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd01c018310 con 0x7fd02c1a3c00 2024-09-12T02:56:24.867 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:56:24.867 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","modified":"2024-09-12T02:56:16.422169Z","created":"2024-09-12T02:54:40.839595Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:56:24.867 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-12T02:56:24.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 msgr2=0x7fd01803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:24.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 0x7fd01803b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0280096c0 tx=0x7fd028031400 comp rx=0 tx=0).stop 2024-09-12T02:56:24.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 msgr2=0x7fd02c19d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 0x7fd02c19d840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00b3d0 tx=0x7fd01c00b890 comp rx=0 tx=0).stop 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 shutdown_connections 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd018038e00 0x7fd01803b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd02c1a3c00 0x7fd02c19d840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 --2- 172.21.15.204:0/3624852261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd02c076720 0x7fd02c1a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 >> 172.21.15.204:0/3624852261 conn(0x7fd02c0ff510 msgr2=0x7fd02c0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 shutdown_connections 2024-09-12T02:56:24.870 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:24.869+0000 7fd033622640 1 -- 172.21.15.204:0/3624852261 wait complete. 2024-09-12T02:56:25.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:25 smithi181 systemd[1]: Started Ceph mon.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:56:26.304 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:56:26.304 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:56:26.558 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.534+0000 7ff344492640 1 -- 172.21.15.204:0/1755688843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 msgr2=0x7ff3240059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.534+0000 7ff344492640 1 --2- 172.21.15.204:0/1755688843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff3240059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff33c06aa00 tx=0x7ff33402c3f0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 -- 172.21.15.204:0/1755688843 shutdown_connections 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 --2- 172.21.15.204:0/1755688843 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff3240059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 -- 172.21.15.204:0/1755688843 >> 172.21.15.204:0/1755688843 conn(0x7ff33c0fc220 msgr2=0x7ff33c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:30.536 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 -- 172.21.15.204:0/1755688843 shutdown_connections 2024-09-12T02:56:30.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 -- 172.21.15.204:0/1755688843 wait complete. 2024-09-12T02:56:30.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 Processor -- start 2024-09-12T02:56:30.537 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.535+0000 7ff344492640 1 -- start start 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff33c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c109780 con 0x7ff324003530 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c109600 con 0x7ff33c19c160 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff344492640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c109900 con 0x7ff33c197890 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57130/0 (socket says 172.21.15.204:57130) 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 -- 172.21.15.204:0/715513389 learned_addr learned my addr 172.21.15.204:0/715513389 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff33c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.538 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff342a2f640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:30.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: mon.a calling monitor election 2024-09-12T02:56:30.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: mon.c calling monitor election 2024-09-12T02:56:30.539 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: monmap epoch 3 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:30.540 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: min_mon_release 19 (squid) 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: election_strategy: 1 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: fsmap 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: mgrmap e13: x(active, since 50s) 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: overall HEALTH_OK 2024-09-12T02:56:30.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.542 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.542 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.542 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 msgr2=0x7ff33c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 msgr2=0x7ff33c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff341a2d640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff342a2f640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 msgr2=0x7ff33c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff342a2f640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 msgr2=0x7ff33c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff342a2f640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff342a2f640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 msgr2=0x7ff33c19bc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.544 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 msgr2=0x7ff33c19c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.536+0000 7ff34222e640 1 -- 172.21.15.204:0/715513389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c1a3ed0 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.537+0000 7ff34222e640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff33c197350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff334005f10 tx=0x7ff3340048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.537+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334012450 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.537+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff334042070 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.537+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c1a40a0 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.537+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33403d8f0 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.538+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a45b0 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.538+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff33403da90 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.538+0000 7ff33b7fe640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 0x7ff32803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.538+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff334038e40 con 0x7ff324003530 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.538+0000 7ff341a2d640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 0x7ff32803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.545 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.539+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c10a8b0 con 0x7ff324003530 2024-09-12T02:56:30.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.539+0000 7ff341a2d640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 0x7ff32803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff330009100 tx=0x7ff330008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:30.546 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.544+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff33403b340 con 0x7ff324003530 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: mon.a calling monitor election 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: mon.c calling monitor election 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:30.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: monmap epoch 3 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:30.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: min_mon_release 19 (squid) 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: election_strategy: 1 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: fsmap 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: mgrmap e13: x(active, since 50s) 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: overall HEALTH_OK 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:30.811 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.811+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff33c1a47e0 con 0x7ff324003530 2024-09-12T02:56:30.811 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.811+0000 7ff33b7fe640 1 -- 172.21.15.204:0/715513389 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7ff334045980 con 0x7ff324003530 2024-09-12T02:56:30.812 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:56:30.812 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","modified":"2024-09-12T02:56:25.198703Z","created":"2024-09-12T02:54:40.839595Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:3300","nonce":0},{"type":"v1","addr":"172.21.15.57:6789","nonce":0}]},"addr":"172.21.15.57:6789/0","public_addr":"172.21.15.57:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:3300","nonce":0},{"type":"v1","addr":"172.21.15.181:6789","nonce":0}]},"addr":"172.21.15.181:6789/0","public_addr":"172.21.15.181:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:56:30.812 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 msgr2=0x7ff32803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 0x7ff32803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff330009100 tx=0x7ff330008040 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 msgr2=0x7ff33c197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff33c197350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff334005f10 tx=0x7ff3340048d0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 shutdown_connections 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff3280394f0 0x7ff32803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff33c19c160 0x7ff33c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff33c197890 0x7ff33c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 --2- 172.21.15.204:0/715513389 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff324003530 0x7ff33c197350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.814 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.813+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 >> 172.21.15.204:0/715513389 conn(0x7ff33c0fc220 msgr2=0x7ff33c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:30.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.814+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 shutdown_connections 2024-09-12T02:56:30.815 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:30.814+0000 7ff344492640 1 -- 172.21.15.204:0/715513389 wait complete. 2024-09-12T02:56:31.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:31 smithi204 ceph-mon[35013]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:31.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:31.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:31 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/715513389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:31.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:31.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:31 smithi057 ceph-mon[30361]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:31.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:31.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:31 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/715513389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:31.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.129 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T02:56:32.130 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T02:56:32.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: mon.a calling monitor election 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: mon.c calling monitor election 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: monmap epoch 3 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: min_mon_release 19 (squid) 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: election_strategy: 1 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:32.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: fsmap 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: mgrmap e13: x(active, since 50s) 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: overall HEALTH_OK 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/715513389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:32.510 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:33.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 -- 172.21.15.57:0/1417392150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e80745c0 msgr2=0x7f89e80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 --2- 172.21.15.57:0/1417392150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e80745c0 0x7f89e80749a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009880 tx=0x7f89d802a670 comp rx=0 tx=0).stop 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 -- 172.21.15.57:0/1417392150 shutdown_connections 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 --2- 172.21.15.57:0/1417392150 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e80745c0 0x7f89e80749a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 -- 172.21.15.57:0/1417392150 >> 172.21.15.57:0/1417392150 conn(0x7f89e80fd290 msgr2=0x7f89e80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 -- 172.21.15.57:0/1417392150 shutdown_connections 2024-09-12T02:56:33.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.184+0000 7f89ef000640 1 -- 172.21.15.57:0/1417392150 wait complete. 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.185+0000 7f89ef000640 1 Processor -- start 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.185+0000 7f89ef000640 1 -- start start 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80745c0 0x7f89e8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e819e410 0x7f89e819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f89e810de60 con 0x7f89e8199b40 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f89e810dce0 con 0x7f89e819e410 2024-09-12T02:56:33.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ef000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f89e810dfe0 con 0x7f89e80745c0 2024-09-12T02:56:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46548/0 (socket says 172.21.15.57:46548) 2024-09-12T02:56:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89dfe00640 1 -- 172.21.15.57:0/761479837 learned_addr learned my addr 172.21.15.57:0/761479837 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89e7e00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80745c0 0x7f89e8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.186+0000 7f89ed000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e819e410 0x7f89e819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 -- 172.21.15.57:0/761479837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80745c0 msgr2=0x7f89e8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80745c0 0x7f89e8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 -- 172.21.15.57:0/761479837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e819e410 msgr2=0x7f89e819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e819e410 0x7f89e819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 -- 172.21.15.57:0/761479837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e81a6180 con 0x7f89e8199b40 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89dfe00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89d000b520 tx=0x7f89d000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0004230 con 0x7f89e8199b40 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89d00043d0 con 0x7f89e8199b40 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.187+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d0003d60 con 0x7f89e8199b40 2024-09-12T02:56:33.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.188+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e81a63b0 con 0x7f89e8199b40 2024-09-12T02:56:33.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.188+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e81a6870 con 0x7f89e8199b40 2024-09-12T02:56:33.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.189+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e8075c70 con 0x7f89e8199b40 2024-09-12T02:56:33.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.189+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f89d000e070 con 0x7f89e8199b40 2024-09-12T02:56:33.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.190+0000 7f89e5600640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 0x7f89cc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.190+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f89d004b990 con 0x7f89e8199b40 2024-09-12T02:56:33.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.190+0000 7f89e7e00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 0x7f89cc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.191+0000 7f89e7e00640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 0x7f89cc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009230 tx=0x7f89d8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:33.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.199+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89d001d030 con 0x7f89e8199b40 2024-09-12T02:56:33.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mon.b calling monitor election 2024-09-12T02:56:33.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:33.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.287 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mon.b calling monitor election 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mon.c calling monitor election 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mon.a calling monitor election 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: monmap epoch 3 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: min_mon_release 19 (squid) 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: election_strategy: 1 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:33.288 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: fsmap 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: mgrmap e13: x(active, since 52s) 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: overall HEALTH_OK 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:33.289 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:33.290 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:33.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.387+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f89e81a6bf0 con 0x7f89e8199b40 2024-09-12T02:56:33.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.387+0000 7f89e5600640 1 -- 172.21.15.57:0/761479837 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f89d0014020 con 0x7f89e8199b40 2024-09-12T02:56:33.388 INFO:teuthology.orchestra.run.smithi057.stdout:# minimal ceph.conf for 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:33.388 INFO:teuthology.orchestra.run.smithi057.stdout:[global] 2024-09-12T02:56:33.389 INFO:teuthology.orchestra.run.smithi057.stdout: fsid = 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:33.389 INFO:teuthology.orchestra.run.smithi057.stdout: mon_host = [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-12T02:56:33.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.389+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 msgr2=0x7f89cc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.389+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 0x7f89cc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009230 tx=0x7f89d8005a40 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.389+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 msgr2=0x7f89e819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.389+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f89d000b520 tx=0x7f89d000b9e0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 shutdown_connections 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f89cc0394f0 0x7f89cc03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e819e410 0x7f89e819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f89e8199b40 0x7f89e819ded0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 --2- 172.21.15.57:0/761479837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e80745c0 0x7f89e8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 >> 172.21.15.57:0/761479837 conn(0x7f89e80fd290 msgr2=0x7f89e81078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:33.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 shutdown_connections 2024-09-12T02:56:33.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:33.390+0000 7f89ef000640 1 -- 172.21.15.57:0/761479837 wait complete. 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mon.b calling monitor election 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mon.b calling monitor election 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mon.c calling monitor election 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mon.a calling monitor election 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: monmap epoch 3 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:33.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: min_mon_release 19 (squid) 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: election_strategy: 1 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: fsmap 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: mgrmap e13: x(active, since 52s) 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: overall HEALTH_OK 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:33.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:33.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mon.b calling monitor election 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mon.b calling monitor election 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mon.c calling monitor election 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mon.a calling monitor election 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: monmap epoch 3 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: last_changed 2024-09-12T02:56:25.198703+0000 2024-09-12T02:56:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: created 2024-09-12T02:54:40.839595+0000 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: min_mon_release 19 (squid) 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: election_strategy: 1 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: 0: [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] mon.a 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: 2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: fsmap 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: mgrmap e13: x(active, since 52s) 2024-09-12T02:56:33.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: overall HEALTH_OK 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:34.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 02:56:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T02:56:34.198+0000 7f504f000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T02:56:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:34 smithi057 ceph-mon[30361]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:56:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:34 smithi057 ceph-mon[30361]: Reconfiguring daemon mon.a on smithi057 2024-09-12T02:56:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:34 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/761479837' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:34.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:34 smithi204 ceph-mon[35013]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:56:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:34 smithi204 ceph-mon[35013]: Reconfiguring daemon mon.a on smithi057 2024-09-12T02:56:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:34 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/761479837' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:34 smithi181 ceph-mon[35320]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:56:34.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:34 smithi181 ceph-mon[35320]: Reconfiguring daemon mon.a on smithi057 2024-09-12T02:56:34.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:56:34.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:34 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/761479837' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:34.551 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T02:56:34.552 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:56:34.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:56:34.610 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:56:34.611 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:34.707 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:56:34.707 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:56:34.735 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:56:34.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:34.800 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:56:34.800 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:56:34.833 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:56:34.833 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:56:34.902 INFO:tasks.cephadm:Adding mgr.x on smithi057 2024-09-12T02:56:34.903 INFO:tasks.cephadm:Adding mgr.y on smithi181 2024-09-12T02:56:34.903 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-09-12T02:56:34.903 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi057=x;smithi181=y;smithi204=z' 2024-09-12T02:56:35.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:35.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:35.189 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:35.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:35.351 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:35.384 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:35.384 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.384 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.384 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:35.385 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:35.385 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:35.385 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 -- 172.21.15.204:0/2747037056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc1005f0 msgr2=0x7ffafc1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2747037056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc1005f0 0x7ffafc1009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009c20 tx=0x7ffaec02aa90 comp rx=0 tx=0).stop 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 -- 172.21.15.204:0/2747037056 shutdown_connections 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2747037056 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc1005f0 0x7ffafc1009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 -- 172.21.15.204:0/2747037056 >> 172.21.15.204:0/2747037056 conn(0x7ffafc0fbda0 msgr2=0x7ffafc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 -- 172.21.15.204:0/2747037056 shutdown_connections 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.597+0000 7ffb040bc640 1 -- 172.21.15.204:0/2747037056 wait complete. 2024-09-12T02:56:35.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 Processor -- start 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 -- start start 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffafc1005f0 0x7ffafc18d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc18bf60 0x7ffafc18c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc106700 con 0x7ffafc18bf60 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc106580 con 0x7ffafc1005f0 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb040bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc106880 con 0x7ffafc18c920 2024-09-12T02:56:35.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb02659640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb02659640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:57142/0 (socket says 172.21.15.204:57142) 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb02659640 1 -- 172.21.15.204:0/2629641655 learned_addr learned my addr 172.21.15.204:0/2629641655 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb01e58640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffafc1005f0 0x7ffafc18d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.598+0000 7ffb01657640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc18bf60 0x7ffafc18c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 -- 172.21.15.204:0/2629641655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffafc1005f0 msgr2=0x7ffafc18d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffafc1005f0 0x7ffafc18d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 -- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc18bf60 msgr2=0x7ffafc18c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc18bf60 0x7ffafc18c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 -- 172.21.15.204:0/2629641655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc1a15d0 con 0x7ffafc18c920 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb02659640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b450 tx=0x7ffaf400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4015020 con 0x7ffafc18c920 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1a18c0 con 0x7ffafc18c920 2024-09-12T02:56:35.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf4011d70 con 0x7ffafc18c920 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400f340 con 0x7ffafc18c920 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.599+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc1a1e30 con 0x7ffafc18c920 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.600+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ffaf4018020 con 0x7ffafc18c920 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.600+0000 7ffafaffd640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 0x7ffae403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.600+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc1a20c0 con 0x7ffafc18c920 2024-09-12T02:56:35.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.600+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffaf404bdf0 con 0x7ffafc18c920 2024-09-12T02:56:35.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.600+0000 7ffb01e58640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 0x7ffae403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.601+0000 7ffb01e58640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 0x7ffae403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffaec005e30 tx=0x7ffaec03a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:35.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.605+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaf401ae20 con 0x7ffafc18c920 2024-09-12T02:56:35.782 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.781+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7ffafc0fd610 con 0x7ffae40394f0 2024-09-12T02:56:35.788 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.788+0000 7ffafaffd640 1 -- 172.21.15.204:0/2629641655 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffafc0fd610 con 0x7ffae40394f0 2024-09-12T02:56:35.788 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 msgr2=0x7ffae403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 0x7ffae403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffaec005e30 tx=0x7ffaec03a040 comp rx=0 tx=0).stop 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 msgr2=0x7ffafc1a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b450 tx=0x7ffaf400b910 comp rx=0 tx=0).stop 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 shutdown_connections 2024-09-12T02:56:35.791 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ffae40394f0 0x7ffae403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffafc18c920 0x7ffafc1a1040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffafc18bf60 0x7ffafc18c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 --2- 172.21.15.204:0/2629641655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffafc1005f0 0x7ffafc18d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 >> 172.21.15.204:0/2629641655 conn(0x7ffafc0fbda0 msgr2=0x7ffafc0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 shutdown_connections 2024-09-12T02:56:35.792 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:56:35.790+0000 7ffb040bc640 1 -- 172.21.15.204:0/2629641655 wait complete. 2024-09-12T02:56:36.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:36 smithi204 ceph-mon[35013]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:56:36.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:36 smithi204 ceph-mon[35013]: Reconfiguring daemon mon.b on smithi181 2024-09-12T02:56:36.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:36.336 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:36 smithi181 ceph-mon[35320]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:56:36.336 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:36 smithi181 ceph-mon[35320]: Reconfiguring daemon mon.b on smithi181 2024-09-12T02:56:36.336 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:36.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:36 smithi057 ceph-mon[30361]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:56:36.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:36 smithi057 ceph-mon[30361]: Reconfiguring daemon mon.b on smithi181 2024-09-12T02:56:36.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:36.424 DEBUG:teuthology.orchestra.run.smithi181:mgr.y> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T02:56:36.429 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T02:56:36.433 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T02:56:36.434 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T02:56:36.434 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:56:36.473 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:56:36.473 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_1 2024-09-12T02:56:36.551 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-09-12 02:55:52.817150330 +0000 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-09-12 02:51:34.013897985 +0000 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-09-12 02:51:34.013897985 +0000 2024-09-12T02:56:36.552 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-09-12 02:51:34.013897985 +0000 2024-09-12T02:56:36.552 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:56:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-09-12T02:56:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-09-12T02:56:36.649 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000379247 s, 1.4 MB/s 2024-09-12T02:56:36.651 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:56:36.689 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_2 2024-09-12T02:56:36.765 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:56:36.765 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:36.765 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-12T02:56:36.765 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:36.766 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:36.766 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-09-12 02:55:52.857149680 +0000 2024-09-12T02:56:36.766 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-09-12 02:51:34.489890345 +0000 2024-09-12T02:56:36.766 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-09-12 02:51:34.489890345 +0000 2024-09-12T02:56:36.766 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-09-12 02:51:34.489890345 +0000 2024-09-12T02:56:36.766 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:56:36.859 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-09-12T02:56:36.859 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-09-12T02:56:36.859 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000363724 s, 1.4 MB/s 2024-09-12T02:56:36.861 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:56:36.898 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_3 2024-09-12T02:56:36.973 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:56:36.973 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:36.973 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-12T02:56:36.973 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:36.974 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:36.974 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-09-12 02:55:52.895149062 +0000 2024-09-12T02:56:36.974 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-09-12 02:51:34.973882573 +0000 2024-09-12T02:56:36.974 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-09-12 02:51:34.973882573 +0000 2024-09-12T02:56:36.974 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-09-12 02:51:34.973882573 +0000 2024-09-12T02:56:36.974 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:56:37.066 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-09-12T02:56:37.066 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-09-12T02:56:37.066 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000359325 s, 1.4 MB/s 2024-09-12T02:56:37.069 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:56:37.105 DEBUG:teuthology.orchestra.run.smithi057:> stat /dev/vg_nvme/lv_4 2024-09-12T02:56:37.180 INFO:teuthology.orchestra.run.smithi057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:56:37.180 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:37.180 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-09-12 02:55:52.933148444 +0000 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-09-12 02:51:35.449874930 +0000 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-09-12 02:51:35.449874930 +0000 2024-09-12T02:56:37.181 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-09-12 02:51:35.449874930 +0000 2024-09-12T02:56:37.181 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:56:37.273 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records in 2024-09-12T02:56:37.274 INFO:teuthology.orchestra.run.smithi057.stderr:1+0 records out 2024-09-12T02:56:37.274 INFO:teuthology.orchestra.run.smithi057.stderr:512 bytes copied, 0.000376752 s, 1.4 MB/s 2024-09-12T02:56:37.276 DEBUG:teuthology.orchestra.run.smithi057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:56:37.312 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-12T02:56:37.313 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:56:37.332 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:56:37.332 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-12 02:56:24.643239572 +0000 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-12 02:51:10.709653253 +0000 2024-09-12T02:56:37.391 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-12 02:51:10.709653253 +0000 2024-09-12T02:56:37.392 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-12 02:51:10.709653253 +0000 2024-09-12T02:56:37.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:56:37.459 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-12T02:56:37.459 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-12T02:56:37.459 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000287548 s, 1.8 MB/s 2024-09-12T02:56:37.460 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:56:37.520 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-12 02:56:24.643239572 +0000 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-12 02:51:11.066645531 +0000 2024-09-12T02:56:37.578 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-12 02:51:11.066645531 +0000 2024-09-12T02:56:37.579 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-12 02:51:11.066645531 +0000 2024-09-12T02:56:37.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:56:37.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: Saving service mgr spec with placement smithi057=x;smithi181=y;smithi204=z;count:3 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:37 smithi057 ceph-mon[30361]: Reconfiguring daemon mon.c on smithi204 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: Saving service mgr spec with placement smithi057=x;smithi181=y;smithi204=z;count:3 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:56:37.641 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:37.642 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:37.642 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:37.642 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:37 smithi181 ceph-mon[35320]: Reconfiguring daemon mon.c on smithi204 2024-09-12T02:56:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi057=x;smithi181=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: Saving service mgr spec with placement smithi057=x;smithi181=y;smithi204=z;count:3 2024-09-12T02:56:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:37.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:37 smithi204 ceph-mon[35013]: Reconfiguring daemon mon.c on smithi204 2024-09-12T02:56:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-12T02:56:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-12T02:56:37.645 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000232611 s, 2.2 MB/s 2024-09-12T02:56:37.645 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:56:37.704 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-09-12T02:56:37.760 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:56:37.760 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-12 02:56:24.644239550 +0000 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-12 02:51:11.418637918 +0000 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-12 02:51:11.418637918 +0000 2024-09-12T02:56:37.761 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-12 02:51:11.418637918 +0000 2024-09-12T02:56:37.761 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:56:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-12T02:56:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-12T02:56:37.826 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000186889 s, 2.7 MB/s 2024-09-12T02:56:37.827 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:56:37.884 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-12 02:56:24.644239550 +0000 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-12 02:51:11.747630801 +0000 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-12 02:51:11.747630801 +0000 2024-09-12T02:56:37.940 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2024-09-12 02:51:11.747630801 +0000 2024-09-12T02:56:37.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:56:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-12T02:56:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-12T02:56:38.003 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000214146 s, 2.4 MB/s 2024-09-12T02:56:38.005 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:56:38.061 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-12T02:56:38.062 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:56:38.078 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:56:38.078 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-12 02:56:15.754316353 +0000 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-12 02:51:05.490143951 +0000 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-12 02:51:05.490143951 +0000 2024-09-12T02:56:38.134 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-12 02:51:05.490143951 +0000 2024-09-12T02:56:38.135 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:56:38.199 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-12T02:56:38.199 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-12T02:56:38.199 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000213676 s, 2.4 MB/s 2024-09-12T02:56:38.201 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:56:38.257 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-12 02:56:15.754316353 +0000 2024-09-12T02:56:38.314 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-12 02:51:05.827139207 +0000 2024-09-12T02:56:38.315 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-12 02:51:05.827139207 +0000 2024-09-12T02:56:38.315 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-12 02:51:05.827139207 +0000 2024-09-12T02:56:38.315 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:56:38.379 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-12T02:56:38.379 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-12T02:56:38.379 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000222867 s, 2.3 MB/s 2024-09-12T02:56:38.380 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:56:38.437 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-09-12T02:56:38.492 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:56:38.492 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:38.492 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T02:56:38.492 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:38.493 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:38.493 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-12 02:56:15.755316339 +0000 2024-09-12T02:56:38.493 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-12 02:51:06.146134717 +0000 2024-09-12T02:56:38.493 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-12 02:51:06.146134717 +0000 2024-09-12T02:56:38.493 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-12 02:51:06.146134717 +0000 2024-09-12T02:56:38.493 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:56:38.555 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-12T02:56:38.555 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-12T02:56:38.555 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000198224 s, 2.6 MB/s 2024-09-12T02:56:38.556 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:56:38.613 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-09-12T02:56:38.668 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:56:38.668 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:56:38.668 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-12 02:56:15.755316339 +0000 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-12 02:51:06.468130184 +0000 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-12 02:51:06.468130184 +0000 2024-09-12T02:56:38.669 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-12 02:51:06.468130184 +0000 2024-09-12T02:56:38.669 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:56:38.731 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-12T02:56:38.731 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-12T02:56:38.731 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000179396 s, 2.9 MB/s 2024-09-12T02:56:38.732 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:56:38.789 INFO:tasks.cephadm:Deploying osd.0 on smithi057 with /dev/vg_nvme/lv_4... 2024-09-12T02:56:38.790 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:56:38.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:38.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:38.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:38 smithi181 ceph-mon[35320]: Deploying daemon mgr.z on smithi204 2024-09-12T02:56:39.031 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:39.031 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.031 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.031 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.031 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:39.032 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:38 smithi204 ceph-mon[35013]: Deploying daemon mgr.z on smithi204 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:39.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:39.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:39.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:38 smithi057 ceph-mon[30361]: Deploying daemon mgr.z on smithi204 2024-09-12T02:56:39.165 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:39.644 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 systemd[1]: Starting Ceph mgr.z for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:56:39.904 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 podman[36802]: 2024-09-12 02:56:39.650537804 +0000 UTC m=+0.081355012 container create 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T02:56:39.905 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 podman[36802]: 2024-09-12 02:56:39.585254438 +0000 UTC m=+0.016071658 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:56:39.905 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 podman[36802]: 2024-09-12 02:56:39.878007722 +0000 UTC m=+0.308824946 container init 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T02:56:39.905 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 podman[36802]: 2024-09-12 02:56:39.880602264 +0000 UTC m=+0.311419474 container start 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 ceph-mgr[36821]: -- 172.21.15.204:0/2808028681 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5562db2dd0e0 con 0x5562db2ba800 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 bash[36802]: 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:39 smithi204 systemd[1]: Started Ceph mgr.z for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.005+0000 7ff2005d4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.064+0000 7ff2005d4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:56:40.211 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.209+0000 7ff2005d4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:56:40.558 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.274+0000 7ff2005d4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:56:40.558 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.419+0000 7ff2005d4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:56:40.558 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.557+0000 7ff2005d4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.868+0000 7ff2005d4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:40.928+0000 7ff2005d4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:41.111+0000 7ff2005d4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:41.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-mon[35013]: Deploying daemon mgr.y on smithi181 2024-09-12T02:56:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:41 smithi181 ceph-mon[35320]: Deploying daemon mgr.y on smithi181 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:41.341 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:41 smithi057 ceph-mon[30361]: Deploying daemon mgr.y on smithi181 2024-09-12T02:56:42.113 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:41 smithi181 systemd[1]: Starting Ceph mgr.y for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T02:56:42.287 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:56:42.320 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_4 2024-09-12T02:56:42.373 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 podman[36666]: 2024-09-12 02:56:42.015493561 +0000 UTC m=+0.015717553 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:56:42.374 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 podman[36666]: 2024-09-12 02:56:42.135937865 +0000 UTC m=+0.136161896 container create 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T02:56:42.374 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 podman[36666]: 2024-09-12 02:56:42.346569814 +0000 UTC m=+0.346793809 container init 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T02:56:42.374 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 podman[36666]: 2024-09-12 02:56:42.34929313 +0000 UTC m=+0.349517125 container start 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:56:42.393 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:41.994+0000 7ff2005d4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:56:42.393 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:42.061+0000 7ff2005d4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-mgr[36684]: -- 172.21.15.181:0/1462446577 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55835e6350e0 con 0x55835e613000 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 bash[36666]: 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 systemd[1]: Started Ceph mgr.y for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:42.475+0000 7fe65097a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:42.535+0000 7fe65097a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:56:42.689 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:42.687+0000 7fe65097a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:56:42.699 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:42.953 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:42.757+0000 7fe65097a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:56:42.953 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:42.909+0000 7fe65097a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:56:43.074 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:42.789+0000 7ff2005d4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:56:43.074 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:42.910+0000 7ff2005d4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:56:43.074 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.072+0000 7ff2005d4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:56:43.364 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.134+0000 7ff2005d4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:56:43.364 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.289+0000 7ff2005d4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:56:43.364 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.362+0000 7ff2005d4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:56:43.373 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:43.070+0000 7fe65097a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 -- 172.21.15.57:0/2023341723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8103c50 msgr2=0x7f5cd8104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 --2- 172.21.15.57:0/2023341723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8103c50 0x7f5cd8104050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009900 tx=0x7f5cc802a3f0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 -- 172.21.15.57:0/2023341723 shutdown_connections 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 --2- 172.21.15.57:0/2023341723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8105810 0x7f5cd810c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 --2- 172.21.15.57:0/2023341723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 0x7f5cd81052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 --2- 172.21.15.57:0/2023341723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8103c50 0x7f5cd8104050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 -- 172.21.15.57:0/2023341723 >> 172.21.15.57:0/2023341723 conn(0x7f5cd80ff3e0 msgr2=0x7f5cd8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:43.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 -- 172.21.15.57:0/2023341723 shutdown_connections 2024-09-12T02:56:43.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.383+0000 7f5cdd200640 1 -- 172.21.15.57:0/2023341723 wait complete. 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.384+0000 7f5cdd200640 1 Processor -- start 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.384+0000 7f5cdd200640 1 -- start start 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 0x7f5cd819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8105810 0x7f5cd81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810f8b0 con 0x7f5cd8104e50 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810f730 con 0x7f5cd8105810 2024-09-12T02:56:43.386 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cdd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810fa30 con 0x7f5cd8103c50 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:40676/0 (socket says 172.21.15.57:40676) 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7400640 1 -- 172.21.15.57:0/4213396105 learned_addr learned my addr 172.21.15.57:0/4213396105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd6a00640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 0x7f5cd819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7e00640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8105810 0x7f5cd81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7400640 1 -- 172.21.15.57:0/4213396105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8105810 msgr2=0x7f5cd81a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.385+0000 7f5cd7400640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8105810 0x7f5cd81a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cd7400640 1 -- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 msgr2=0x7f5cd819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cd7400640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 0x7f5cd819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cd7400640 1 -- 172.21.15.57:0/4213396105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd81a1b70 con 0x7f5cd8103c50 2024-09-12T02:56:43.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cd7400640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8031610 tx=0x7f5cc8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc803f030 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8003ad0 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8035070 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd81a1e00 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.386+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd81a2250 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.387+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5cc8042030 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.387+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd810b0a0 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.388+0000 7f5ccfe00640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 0x7f5cbc03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.388+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5cc806ca90 con 0x7f5cd8103c50 2024-09-12T02:56:43.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.388+0000 7f5cd6a00640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 0x7f5cbc03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.389+0000 7f5cd6a00640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 0x7f5cbc03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cd819b460 tx=0x7f5cc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:43.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.393+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cc803ae50 con 0x7f5cd8103c50 2024-09-12T02:56:43.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:43.587+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5cd8062630 con 0x7f5cbc0392c0 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:43.643 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.423+0000 7ff2005d4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:56:43.643 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.585+0000 7ff2005d4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:43.646 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:43.394+0000 7fe65097a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:56:43.646 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:43.455+0000 7fe65097a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:43.647 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:43.931 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.659+0000 7ff2005d4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:56:43.932 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.725+0000 7ff2005d4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:56:43.932 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.807+0000 7ff2005d4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:56:43.932 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.869+0000 7ff2005d4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:56:43.932 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:43.930+0000 7ff2005d4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:56:43.953 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:43.645+0000 7fe65097a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:44.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:44.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:44.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.893 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T02:56:44.712+0000 7ff2005d4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:44.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:44.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:44.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:44.537+0000 7fe65097a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:44.605+0000 7fe65097a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:56:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:56:44.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:45.577 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:45.577 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:56:45.577 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: Reconfiguring daemon mgr.x on smithi057 2024-09-12T02:56:45.577 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: Standby manager daemon z started 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.319+0000 7fe65097a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.435+0000 7fe65097a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:56:45.578 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.576+0000 7fe65097a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:56:45.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:45.696+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f5cc8003c80 con 0x7f5cd8103c50 2024-09-12T02:56:45.855 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.637+0000 7fe65097a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:56:45.855 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.786+0000 7fe65097a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:56:45.855 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.854+0000 7fe65097a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: Reconfiguring daemon mgr.x on smithi057 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: Standby manager daemon z started 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:56:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:45 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: Reconfiguring daemon mgr.x on smithi057 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: Standby manager daemon z started 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:56:45.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:45 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/1468548017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:46.135 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:45.914+0000 7fe65097a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:56:46.135 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.071+0000 7fe65097a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:56:46.135 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.134+0000 7fe65097a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:56:46.401 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.200+0000 7fe65097a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:56:46.401 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.283+0000 7fe65097a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:56:46.401 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.342+0000 7fe65097a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:56:46.401 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:46.401+0000 7fe65097a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:46.888 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-12T02:56:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:56:46.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:46.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:46.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:47.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:47.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: mgrmap e14: x(active, since 66s), standbys: z 2024-09-12T02:56:47.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:56:47.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:47.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:47.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:47.453 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T02:56:47.188+0000 7fe65097a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:56:47.719 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:56:47.718+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5cc80035f0 con 0x7f5cd8103c50 2024-09-12T02:56:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: Standby manager daemon y started 2024-09-12T02:56:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:56:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:56:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: Standby manager daemon y started 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: Standby manager daemon y started 2024-09-12T02:56:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/1016783121' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:48.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]: dispatch 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]': finished 2024-09-12T02:56:48.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:56:48.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:56:48.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1254155268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]: dispatch 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]': finished 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:56:49.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1254155268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:49.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:49.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: mgrmap e15: x(active, since 68s), standbys: z, y 2024-09-12T02:56:49.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:56:49.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]: dispatch 2024-09-12T02:56:49.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/170357656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32912ece-df23-430a-95fe-259e65d98666"}]': finished 2024-09-12T02:56:49.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:56:49.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:56:49.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1254155268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:56:51.005 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:50 smithi181 ceph-mon[35320]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:51.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:50 smithi057 ceph-mon[30361]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:51.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:50 smithi204 ceph-mon[35013]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:52.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:52 smithi057 ceph-mon[30361]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:53.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:52 smithi204 ceph-mon[35013]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:53.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:52 smithi181 ceph-mon[35320]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:54.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:54 smithi057 ceph-mon[30361]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:54 smithi204 ceph-mon[35013]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:54 smithi181 ceph-mon[35320]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:56.844 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:56 smithi057 ceph-mon[30361]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:56 smithi204 ceph-mon[35013]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:56 smithi181 ceph-mon[35320]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:57.768 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:56:57.768 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:57.768 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:57 smithi057 ceph-mon[30361]: Deploying daemon osd.0 on smithi057 2024-09-12T02:56:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:56:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:57 smithi204 ceph-mon[35013]: Deploying daemon osd.0 on smithi057 2024-09-12T02:56:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:56:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:57 smithi181 ceph-mon[35320]: Deploying daemon osd.0 on smithi057 2024-09-12T02:56:59.000 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:56:58 smithi057 ceph-mon[30361]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:56:58 smithi204 ceph-mon[35013]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:56:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:56:58 smithi181 ceph-mon[35320]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:00.942 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:00 smithi057 ceph-mon[30361]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:00 smithi204 ceph-mon[35013]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:00 smithi181 ceph-mon[35320]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:03.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:02 smithi057 ceph-mon[30361]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:02 smithi204 ceph-mon[35013]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:03.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:02 smithi181 ceph-mon[35320]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:04.858 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:04 smithi057 ceph-mon[30361]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:04 smithi204 ceph-mon[35013]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:04 smithi181 ceph-mon[35320]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:05 smithi057 ceph-mon[30361]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:06.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:06.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:05 smithi204 ceph-mon[35013]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:05 smithi181 ceph-mon[35320]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:08.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:08.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:09.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:09.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:09.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:09.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.851+0000 7f5ccfe00640 1 -- 172.21.15.57:0/4213396105 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5cd8062630 con 0x7f5cbc0392c0 2024-09-12T02:57:09.852 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 0 on host 'smithi057' 2024-09-12T02:57:09.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 msgr2=0x7f5cbc03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 0x7f5cbc03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cd819b460 tx=0x7f5cc4005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:09.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 msgr2=0x7f5cd8199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8031610 tx=0x7f5cc8004270 comp rx=0 tx=0).stop 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 shutdown_connections 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5cbc0392c0 0x7f5cbc03b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8105810 0x7f5cd81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5cd8104e50 0x7f5cd819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 --2- 172.21.15.57:0/4213396105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd8103c50 0x7f5cd8199f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.853+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 >> 172.21.15.57:0/4213396105 conn(0x7f5cd80ff3e0 msgr2=0x7f5cd8108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.854+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 shutdown_connections 2024-09-12T02:57:09.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:57:09.854+0000 7f5cdd200640 1 -- 172.21.15.57:0/4213396105 wait complete. 2024-09-12T02:57:10.670 DEBUG:teuthology.orchestra.run.smithi057:osd.0> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T02:57:10.674 INFO:tasks.cephadm:Deploying osd.1 on smithi181 with /dev/vg_nvme/lv_4... 2024-09-12T02:57:10.674 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:57:10.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:10 smithi057 ceph-mon[30361]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:10.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:10.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:10.937 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:10 smithi181 ceph-mon[35320]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:11.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:10 smithi204 ceph-mon[35013]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:11.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:11.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:11.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:11.214 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:57:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T02:57:11.041+0000 7f7d3a420640 -1 osd.0 0 log_to_monitors true 2024-09-12T02:57:11.962 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:11 smithi181 ceph-mon[35320]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:11.963 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:11 smithi181 ceph-mon[35320]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:11.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:11 smithi057 ceph-mon[30361]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:11.980 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:11 smithi057 ceph-mon[30361]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:11 smithi204 ceph-mon[35013]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:11 smithi204 ceph-mon[35013]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:12 smithi181 ceph-mon[35320]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:13.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:12 smithi057 ceph-mon[30361]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: from='osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:12 smithi204 ceph-mon[35013]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:57:13.322 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:57:13.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_4 2024-09-12T02:57:13.594 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: from='osd.0 ' entity='osd.0' 2024-09-12T02:57:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.870 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T02:57:13.577+0000 7f7d35000640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T02:57:13.870 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:57:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T02:57:13.596+0000 7f7d2b000640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: from='osd.0 ' entity='osd.0' 2024-09-12T02:57:13.939 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/435838678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84102690 msgr2=0x7f0e84108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:14.030 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/435838678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84102690 0x7f0e84108f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009ca0 tx=0x7f0e7c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/435838678 shutdown_connections 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/435838678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84102690 0x7f0e84108f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/435838678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84101cd0 0x7f0e84102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/435838678 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84100ad0 0x7f0e84100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/435838678 >> 172.21.15.181:0/435838678 conn(0x7f0e840fc280 msgr2=0x7f0e840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/435838678 shutdown_connections 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.024+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/435838678 wait complete. 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 Processor -- start 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 -- start start 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 0x7f0e841983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:14.031 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84102690 0x7f0e8419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c750 con 0x7f0e84101cd0 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c5d0 con 0x7f0e84100ad0 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8bd4d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410c8d0 con 0x7f0e84102690 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e89ae9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e89ae9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:46414/0 (socket says 172.21.15.181:46414) 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e89ae9640 1 -- 172.21.15.181:0/438165477 learned_addr learned my addr 172.21.15.181:0/438165477 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e8a2ea640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84102690 0x7f0e8419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.025+0000 7f0e892e8640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 0x7f0e841983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:14.032 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 -- 172.21.15.181:0/438165477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84102690 msgr2=0x7f0e8419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84102690 0x7f0e8419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 -- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 msgr2=0x7f0e841983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 0x7f0e841983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 -- 172.21.15.181:0/438165477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8419fae0 con 0x7f0e84100ad0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e892e8640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 0x7f0e841983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e89ae9640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400d770 tx=0x7f0e7400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74017070 con 0x7f0e84100ad0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e74009c30 con 0x7f0e84100ad0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8419fdd0 con 0x7f0e84100ad0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74003dd0 con 0x7f0e84100ad0 2024-09-12T02:57:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.026+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841a0280 con 0x7f0e84100ad0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.027+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e841a05a0 con 0x7f0e84100ad0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.028+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0e74003fb0 con 0x7f0e84100ad0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.029+0000 7f0e7affd640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 0x7f0e700a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.029+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f0e740ccda0 con 0x7f0e84100ad0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.029+0000 7f0e892e8640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 0x7f0e700a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.030+0000 7f0e892e8640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 0x7f0e700a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e800058c0 tx=0x7f0e80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.033+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e74009de0 con 0x7f0e84100ad0 2024-09-12T02:57:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:57:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:57:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: from='osd.0 ' entity='osd.0' 2024-09-12T02:57:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.211 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:14.210+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0e8406a680 con 0x7f0e700a2ba0 2024-09-12T02:57:14.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:57:14.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:57:14.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] boot 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi057 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 17094M 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:14.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:14.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.875 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] boot 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:14.876 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi057 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 17094M 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:14.877 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: osd.0 [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] boot 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:57:15.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi057 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 17094M 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/2098369867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]': finished 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:57:17.107 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:16 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/2098369867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]': finished 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:57:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:16 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/2098369867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]: dispatch 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ea4ec1-f61c-4839-b2c4-453c0f8e2415"}]': finished 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:57:17.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:16 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:18.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:17 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/667540136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:18.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:17 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/667540136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:17 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/667540136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:19.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:18 smithi057 ceph-mon[30361]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:18 smithi204 ceph-mon[35013]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:19.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:18 smithi181 ceph-mon[35320]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:21.006 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:20 smithi181 ceph-mon[35320]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:20 smithi057 ceph-mon[30361]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:21.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:20 smithi204 ceph-mon[35013]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:22 smithi181 ceph-mon[35320]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:22 smithi057 ceph-mon[30361]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:22 smithi204 ceph-mon[35013]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:24.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:24 smithi057 ceph-mon[30361]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:25.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:24 smithi204 ceph-mon[35013]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:25.200 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:24 smithi181 ceph-mon[35320]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:26.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:57:26.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:26.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:25 smithi057 ceph-mon[30361]: Deploying daemon osd.1 on smithi181 2024-09-12T02:57:26.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:57:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:25 smithi204 ceph-mon[35013]: Deploying daemon osd.1 on smithi181 2024-09-12T02:57:26.201 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:57:26.201 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:26.201 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:25 smithi181 ceph-mon[35320]: Deploying daemon osd.1 on smithi181 2024-09-12T02:57:27.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:26 smithi057 ceph-mon[30361]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:27.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:26 smithi204 ceph-mon[35013]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:27.178 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:26 smithi181 ceph-mon[35320]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:29.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:28 smithi057 ceph-mon[30361]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:29.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:28 smithi204 ceph-mon[35013]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:29.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:28 smithi181 ceph-mon[35320]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:31.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:30 smithi057 ceph-mon[30361]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:31.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:30 smithi204 ceph-mon[35013]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:31.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:30 smithi181 ceph-mon[35320]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:32.020 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:31 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:32.020 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:31 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:32.020 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:31 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:33.318 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:33 smithi181 ceph-mon[35320]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:33.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:33 smithi057 ceph-mon[30361]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:33.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:33 smithi204 ceph-mon[35013]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:34.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:34.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:35.125 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.124+0000 7f0e7affd640 1 -- 172.21.15.181:0/438165477 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e8406a680 con 0x7f0e700a2ba0 2024-09-12T02:57:35.125 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 1 on host 'smithi181' 2024-09-12T02:57:35.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 msgr2=0x7f0e700a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:35.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 0x7f0e700a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e800058c0 tx=0x7f0e80005850 comp rx=0 tx=0).stop 2024-09-12T02:57:35.127 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 msgr2=0x7f0e84197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400d770 tx=0x7f0e7400dc30 comp rx=0 tx=0).stop 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 shutdown_connections 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0e700a2ba0 0x7f0e700a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e84102690 0x7f0e8419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0e84101cd0 0x7f0e841983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 --2- 172.21.15.181:0/438165477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e84100ad0 0x7f0e84197eb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 >> 172.21.15.181:0/438165477 conn(0x7f0e840fc280 msgr2=0x7f0e84069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 shutdown_connections 2024-09-12T02:57:35.128 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:57:35.127+0000 7f0e8bd4d640 1 -- 172.21.15.181:0/438165477 wait complete. 2024-09-12T02:57:35.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:35 smithi057 ceph-mon[30361]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:35.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:35 smithi204 ceph-mon[35013]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:35.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:35 smithi181 ceph-mon[35320]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:35.786 DEBUG:teuthology.orchestra.run.smithi181:osd.1> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T02:57:35.788 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-09-12T02:57:35.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:57:36.043 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:36.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:36.390 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:36.390 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:36.390 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:37.133 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:57:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T02:57:36.877+0000 7fc4fbf3c640 -1 osd.1 0 log_to_monitors true 2024-09-12T02:57:37.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:37 smithi204 ceph-mon[35013]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:37.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:37 smithi204 ceph-mon[35013]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:37.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:37 smithi204 ceph-mon[35013]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:37 smithi181 ceph-mon[35320]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:37 smithi181 ceph-mon[35320]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:37.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:37 smithi181 ceph-mon[35320]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:37 smithi057 ceph-mon[30361]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:37 smithi057 ceph-mon[30361]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:37 smithi057 ceph-mon[30361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:57:38.371 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:57:38.396 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-09-12T02:57:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:57:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.410 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:57:38.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:38.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:57:38.411 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:38 smithi204 ceph-mon[35013]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:57:38.439 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-mon[35320]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: from='osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:57:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:57:38.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:38 smithi057 ceph-mon[30361]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:57:38.641 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:57:39.083 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 -- 172.21.15.204:0/1298736627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c101cd0 msgr2=0x7f1d3c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.083 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/1298736627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c101cd0 0x7f1d3c102150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d3002a410 comp rx=0 tx=0).stop 2024-09-12T02:57:39.083 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 -- 172.21.15.204:0/1298736627 shutdown_connections 2024-09-12T02:57:39.083 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/1298736627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c102690 0x7f1d3c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.083 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/1298736627 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c101cd0 0x7f1d3c102150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/1298736627 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d3c100ad0 0x7f1d3c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 -- 172.21.15.204:0/1298736627 >> 172.21.15.204:0/1298736627 conn(0x7f1d3c0fc280 msgr2=0x7f1d3c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 -- 172.21.15.204:0/1298736627 shutdown_connections 2024-09-12T02:57:39.084 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.082+0000 7f1d40b2e640 1 -- 172.21.15.204:0/1298736627 wait complete. 2024-09-12T02:57:39.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 Processor -- start 2024-09-12T02:57:39.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 -- start start 2024-09-12T02:57:39.085 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c100ad0 0x7f1d3c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d3c101cd0 0x7f1d3c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c10c7a0 con 0x7f1d3c101cd0 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c10c620 con 0x7f1d3c100ad0 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d40b2e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c10c920 con 0x7f1d3c102690 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d3ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d3ad9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:45524/0 (socket says 172.21.15.204:45524) 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d3ad9d640 1 -- 172.21.15.204:0/3642530501 learned_addr learned my addr 172.21.15.204:0/3642530501 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.083+0000 7f1d3a59c640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c100ad0 0x7f1d3c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 -- 172.21.15.204:0/3642530501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c100ad0 msgr2=0x7f1d3c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c100ad0 0x7f1d3c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 -- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d3c101cd0 msgr2=0x7f1d3c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d3c101cd0 0x7f1d3c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 -- 172.21.15.204:0/3642530501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c19faf0 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d3ad9d640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400eb10 tx=0x7f1d3400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34004b20 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d34004cc0 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c19fde0 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010760 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.084+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c1a02f0 con 0x7f1d3c102690 2024-09-12T02:57:39.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.085+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c107ed0 con 0x7f1d3c102690 2024-09-12T02:57:39.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.087+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f1d34010900 con 0x7f1d3c102690 2024-09-12T02:57:39.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.087+0000 7f1d2b7fe640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 0x7f1d240a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.087+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f1d340ce120 con 0x7f1d3c102690 2024-09-12T02:57:39.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.088+0000 7f1d3a59c640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 0x7f1d240a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.088 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.088+0000 7f1d3a59c640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 0x7f1d240a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c06b050 tx=0x7f1d2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.091 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.091+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d3409c9f0 con 0x7f1d3c102690 2024-09-12T02:57:39.162 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T02:57:38.777+0000 7fc4f86d3640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T02:57:39.162 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:57:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T02:57:38.780+0000 7fc4efcdf640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:57:39.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:39.271+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1d3c062630 con 0x7f1d240a2af0 2024-09-12T02:57:39.272 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:39.272 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.272 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.272 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='osd.1 ' entity='osd.1' 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='osd.1 ' entity='osd.1' 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='osd.1 ' entity='osd.1' 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:39.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:40.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:57:40.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:57:40.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi181 2024-09-12T02:57:40.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] boot 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:57:40.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:40.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:57:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:57:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi181 2024-09-12T02:57:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-12T02:57:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] boot 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:57:40.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi181 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 17088M 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:40.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: osd.1 [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] boot 2024-09-12T02:57:40.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:57:40.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:57:41.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:41 smithi181 ceph-mon[35320]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:41.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:41 smithi181 ceph-mon[35320]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:57:41.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:41 smithi057 ceph-mon[30361]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:41 smithi057 ceph-mon[30361]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:57:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:41 smithi204 ceph-mon[35013]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:57:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:41 smithi204 ceph-mon[35013]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:57:42.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/3853956022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]': finished 2024-09-12T02:57:42.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:57:42.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:57:42.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:42 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/3332558937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:42.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/3853956022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]': finished 2024-09-12T02:57:42.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:57:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:57:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:42 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/3332558937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/3853956022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]: dispatch 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6af3e4b4-81a9-4247-b17e-ab3ca353d09b"}]': finished 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:57:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:42 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/3332558937' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:57:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:43 smithi057 ceph-mon[30361]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:43 smithi204 ceph-mon[35013]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:43 smithi181 ceph-mon[35320]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:45.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:45 smithi204 ceph-mon[35013]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:45.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:45 smithi057 ceph-mon[30361]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:45 smithi181 ceph-mon[35320]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:47.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:47 smithi204 ceph-mon[35013]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:47 smithi057 ceph-mon[30361]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:47 smithi181 ceph-mon[35320]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:49.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:49 smithi204 ceph-mon[35013]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:49.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:57:49.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:49 smithi057 ceph-mon[30361]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:57:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:49 smithi181 ceph-mon[35320]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:57:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:50.821 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:50 smithi204 ceph-mon[35013]: Deploying daemon osd.2 on smithi204 2024-09-12T02:57:50.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:50 smithi057 ceph-mon[30361]: Deploying daemon osd.2 on smithi204 2024-09-12T02:57:50.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:50 smithi181 ceph-mon[35320]: Deploying daemon osd.2 on smithi204 2024-09-12T02:57:51.821 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:51 smithi204 ceph-mon[35013]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:51.854 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:51 smithi181 ceph-mon[35320]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:51 smithi057 ceph-mon[30361]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:53.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:53 smithi057 ceph-mon[30361]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:53.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:53 smithi204 ceph-mon[35013]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:53 smithi181 ceph-mon[35320]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:55 smithi057 ceph-mon[30361]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:55 smithi204 ceph-mon[35013]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:55 smithi181 ceph-mon[35320]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:56.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:56.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:56.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:56.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:56.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:56.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:56.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:57.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:57 smithi204 ceph-mon[35013]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:57.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:57 smithi057 ceph-mon[30361]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:57 smithi181 ceph-mon[35320]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:57:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:57:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:57:59.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:59.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:59.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:57:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:57:59.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.933+0000 7f1d2b7fe640 1 -- 172.21.15.204:0/3642530501 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d3c062630 con 0x7f1d240a2af0 2024-09-12T02:57:59.935 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-09-12T02:57:59.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 msgr2=0x7f1d240a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.935 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 0x7f1d240a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c06b050 tx=0x7f1d2c00b410 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 msgr2=0x7f1d3c19f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400eb10 tx=0x7f1d3400c2d0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 shutdown_connections 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d240a2af0 0x7f1d240a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d3c102690 0x7f1d3c19f230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1d3c101cd0 0x7f1d3c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 --2- 172.21.15.204:0/3642530501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d3c100ad0 0x7f1d3c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 >> 172.21.15.204:0/3642530501 conn(0x7f1d3c0fc280 msgr2=0x7f1d3c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 shutdown_connections 2024-09-12T02:57:59.936 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:57:59.935+0000 7f1d40b2e640 1 -- 172.21.15.204:0/3642530501 wait complete. 2024-09-12T02:58:00.575 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T02:58:00.617 INFO:tasks.cephadm:Deploying osd.3 on smithi057 with /dev/vg_nvme/lv_3... 2024-09-12T02:58:00.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:58:00.998 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:00 smithi057 ceph-mon[30361]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:00 smithi204 ceph-mon[35013]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:00 smithi181 ceph-mon[35320]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:02.394 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T02:58:02.036+0000 7f760fcb4640 -1 osd.2 0 log_to_monitors true 2024-09-12T02:58:02.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:02 smithi057 ceph-mon[30361]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:02.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:02 smithi057 ceph-mon[30361]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:02.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:02 smithi057 ceph-mon[30361]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:03.034 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:02 smithi204 ceph-mon[35013]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:03.035 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:02 smithi204 ceph-mon[35013]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:03.035 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:02 smithi204 ceph-mon[35013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:02 smithi181 ceph-mon[35320]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:02 smithi181 ceph-mon[35320]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:02 smithi181 ceph-mon[35320]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:58:03.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:58:03.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:03.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:58:03.978 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi204 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:58:03.979 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:03 smithi057 ceph-mon[30361]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:58:04.148 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:58:04.148 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:04.148 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:58:04.148 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:04.148 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi204 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:58:04.149 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:03 smithi181 ceph-mon[35320]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:58:04.336 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:58:04.369 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_3 2024-09-12T02:58:04.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi204 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:04.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:04.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:04.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:58:04.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:03 smithi204 ceph-mon[35013]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:58:04.744 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:04.893 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T02:58:04.588+0000 7f760bc37640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T02:58:04.893 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T02:58:04.592+0000 7f7603a57640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:58:05.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:04 smithi057 ceph-mon[30361]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:05.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:04 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.181 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:04 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.182 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:04 smithi057 ceph-mon[30361]: from='osd.2 ' entity='osd.2' 2024-09-12T02:58:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:04 smithi181 ceph-mon[35320]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:04 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:04 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:04 smithi181 ceph-mon[35320]: from='osd.2 ' entity='osd.2' 2024-09-12T02:58:05.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 -- 172.21.15.57:0/2954518756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4103c50 msgr2=0x7f9ee4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 --2- 172.21.15.57:0/2954518756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4103c50 0x7f9ee4104050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802a410 comp rx=0 tx=0).stop 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 -- 172.21.15.57:0/2954518756 shutdown_connections 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 --2- 172.21.15.57:0/2954518756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4105810 0x7f9ee410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 --2- 172.21.15.57:0/2954518756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4104e50 0x7f9ee41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 --2- 172.21.15.57:0/2954518756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4103c50 0x7f9ee4104050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 -- 172.21.15.57:0/2954518756 >> 172.21.15.57:0/2954518756 conn(0x7f9ee40ff3e0 msgr2=0x7f9ee4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 -- 172.21.15.57:0/2954518756 shutdown_connections 2024-09-12T02:58:05.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.297+0000 7f9eeb200640 1 -- 172.21.15.57:0/2954518756 wait complete. 2024-09-12T02:58:05.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.298+0000 7f9eeb200640 1 Processor -- start 2024-09-12T02:58:05.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 -- start start 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4103c50 0x7f9ee4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 0x7f9ee41a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee410f900 con 0x7f9ee4105810 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee410f780 con 0x7f9ee4103c50 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eeb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee410fa80 con 0x7f9ee4104e50 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9ee9e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4103c50 0x7f9ee4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9ee9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9ee9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48654/0 (socket says 172.21.15.57:48654) 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9ee9400640 1 -- 172.21.15.57:0/3154575334 learned_addr learned my addr 172.21.15.57:0/3154575334 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.299+0000 7f9eea800640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 0x7f9ee41a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 -- 172.21.15.57:0/3154575334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4103c50 msgr2=0x7f9ee4199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4103c50 0x7f9ee4199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 -- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 msgr2=0x7f9ee41a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 0x7f9ee41a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 -- 172.21.15.57:0/3154575334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a1b20 con 0x7f9ee4104e50 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9eea800640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 0x7f9ee41a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:05.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9ee9400640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9f0 tx=0x7f9ed400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400cd80 con 0x7f9ee4104e50 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a1e10 con 0x7f9ee4104e50 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.300+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4010910 con 0x7f9ee4104e50 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.301+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40188f0 con 0x7f9ee4104e50 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.301+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a22c0 con 0x7f9ee4104e50 2024-09-12T02:58:05.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.302+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee410afa0 con 0x7f9ee4104e50 2024-09-12T02:58:05.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.304+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f9ed4010ac0 con 0x7f9ee4104e50 2024-09-12T02:58:05.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.305+0000 7f9edea00640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 0x7f9ecc0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.306+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f9ed40cdcc0 con 0x7f9ee4104e50 2024-09-12T02:58:05.306 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.306+0000 7f9ee9e00640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 0x7f9ecc0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.306+0000 7f9ee9e00640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 0x7f9ecc0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802aa00 tx=0x7f9ed80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:05.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.310+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ed409bc80 con 0x7f9ee4104e50 2024-09-12T02:58:05.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-mon[35013]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:05.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:05.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:04 smithi204 ceph-mon[35013]: from='osd.2 ' entity='osd.2' 2024-09-12T02:58:05.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:05.503+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9ee4062630 con 0x7f9ecc0a2b20 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] boot 2024-09-12T02:58:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:58:06.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] boot 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:58:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:58:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: osd.2 [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] boot 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:58:06.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:07 smithi057 ceph-mon[30361]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:07 smithi057 ceph-mon[30361]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:58:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:07 smithi204 ceph-mon[35013]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:07 smithi204 ceph-mon[35013]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:58:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:07 smithi181 ceph-mon[35320]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:58:07.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:07.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:07 smithi181 ceph-mon[35320]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:58:07.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:58:08.830 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:08 smithi057 sudo[48668]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:08.831 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:08 smithi057 sudo[48668]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:08.831 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:08 smithi057 sudo[48668]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:08.831 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:08 smithi057 sudo[48668]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2812238452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]': finished 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:58:08.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2812238452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]': finished 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:58:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2812238452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]: dispatch 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4746f9bf-15b6-4dfb-8e93-8140a669499e"}]': finished 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:58:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:09.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:09 smithi057 sudo[48702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:09.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:09 smithi057 sudo[48702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:09.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:09 smithi057 sudo[48702]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:09.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 02:58:09 smithi057 sudo[48702]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:09.561 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43942]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:09.562 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43942]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:09.562 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43942]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:09.562 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43942]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:09.893 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:09 smithi204 sudo[44045]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:09.893 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:09 smithi204 sudo[44045]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:09.894 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:09 smithi204 sudo[44045]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:09.894 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:09 smithi204 sudo[44045]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:09.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:09 smithi204 ceph-mon[35013]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:09.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:09 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:09.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:09 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1949562153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:09.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:09 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:09 smithi181 ceph-mon[35320]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:09 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:09 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1949562153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:09 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:09.954 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43945]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:09.954 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43945]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:09.954 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43945]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:09.954 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 02:58:09 smithi181 sudo[43945]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:09 smithi057 ceph-mon[30361]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:09 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:09 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1949562153' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:09 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:10.393 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:10 smithi204 sudo[44048]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:10.393 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:10 smithi204 sudo[44048]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:10.394 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:10 smithi204 sudo[44048]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:10.394 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 02:58:10 smithi204 sudo[44048]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:11.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:10 smithi057 sudo[49008]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:11.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:10 smithi057 sudo[49008]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:11.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:10 smithi057 sudo[49008]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:11.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:10 smithi057 sudo[49008]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 sudo[49134]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 sudo[49134]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 sudo[49134]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 sudo[49134]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:11 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:11 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:11 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44051]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44051]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44051]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:11.753 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44051]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:11.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:11.763+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9ed40982f0 con 0x7f9ee4104e50 2024-09-12T02:58:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44054]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44054]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 sudo[44054]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:11 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:11 smithi181 sudo[43948]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:58:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:11 smithi181 sudo[43948]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:11 smithi181 sudo[43948]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:12.249 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 sudo[43948]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:12 smithi057 ceph-mon[30361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:12 smithi204 ceph-mon[35013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 sudo[43951]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:58:12.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 sudo[43951]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:58:12.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 sudo[43951]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:58:12.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 sudo[43951]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:12.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:58:12.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:58:12.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:58:12.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:58:12.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:12 smithi181 ceph-mon[35320]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:58:13.564 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:13 smithi057 ceph-mon[30361]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:13 smithi204 ceph-mon[35013]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:13 smithi181 ceph-mon[35320]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:15.342 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:15 smithi057 ceph-mon[30361]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:15 smithi204 ceph-mon[35013]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:15 smithi181 ceph-mon[35320]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:17.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:17 smithi057 ceph-mon[30361]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T02:58:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:17 smithi204 ceph-mon[35013]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T02:58:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:17 smithi181 ceph-mon[35320]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T02:58:18.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:58:18.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:18.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:58:18.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:18.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:58:18.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:19.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:19 smithi057 ceph-mon[30361]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T02:58:19.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:19 smithi057 ceph-mon[30361]: Deploying daemon osd.3 on smithi057 2024-09-12T02:58:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:19 smithi204 ceph-mon[35013]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T02:58:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:19 smithi204 ceph-mon[35013]: Deploying daemon osd.3 on smithi057 2024-09-12T02:58:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:19 smithi181 ceph-mon[35320]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T02:58:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:19 smithi181 ceph-mon[35320]: Deploying daemon osd.3 on smithi057 2024-09-12T02:58:21.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:21 smithi057 ceph-mon[30361]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T02:58:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:21 smithi204 ceph-mon[35013]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T02:58:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:21 smithi181 ceph-mon[35320]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T02:58:23.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:23 smithi057 ceph-mon[30361]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:58:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:23 smithi204 ceph-mon[35013]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:58:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:23 smithi181 ceph-mon[35320]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:58:25.568 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:25 smithi057 ceph-mon[30361]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:25.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:25 smithi204 ceph-mon[35013]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:25 smithi181 ceph-mon[35320]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:26.575 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:26.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:26.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:26.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:26.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:26.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:27.486 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:27 smithi057 ceph-mon[30361]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:27.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:27 smithi204 ceph-mon[35013]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:27 smithi181 ceph-mon[35320]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:29.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:29.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:29.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.483 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.482+0000 7f9edea00640 1 -- 172.21.15.57:0/3154575334 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ee4062630 con 0x7f9ecc0a2b20 2024-09-12T02:58:30.484 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 3 on host 'smithi057' 2024-09-12T02:58:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 msgr2=0x7f9ecc0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 0x7f9ecc0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802aa00 tx=0x7f9ed80315b0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 msgr2=0x7f9ee419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e9f0 tx=0x7f9ed400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 shutdown_connections 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ecc0a2b20 0x7f9ecc0a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ee4105810 0x7f9ee41a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee4104e50 0x7f9ee419a430 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 --2- 172.21.15.57:0/3154575334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ee4103c50 0x7f9ee4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 >> 172.21.15.57:0/3154575334 conn(0x7f9ee40ff3e0 msgr2=0x7f9ee4108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 shutdown_connections 2024-09-12T02:58:30.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:58:30.485+0000 7f9eeb200640 1 -- 172.21.15.57:0/3154575334 wait complete. 2024-09-12T02:58:30.832 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:30.832 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.832 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:30.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:30 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:30.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:30 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:30.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:30 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:31.257 DEBUG:teuthology.orchestra.run.smithi057:osd.3> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T02:58:31.261 INFO:tasks.cephadm:Deploying osd.4 on smithi181 with /dev/vg_nvme/lv_3... 2024-09-12T02:58:31.262 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:58:31.512 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:58:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:31 smithi057 ceph-mon[30361]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:31.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:31 smithi057 ceph-mon[30361]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:58:31.785 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 02:58:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T02:58:31.500+0000 7fd9cc23c640 -1 osd.3 0 log_to_monitors true 2024-09-12T02:58:31.874 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:31 smithi181 ceph-mon[35320]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:31.874 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:31 smithi181 ceph-mon[35320]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:58:31.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:31 smithi204 ceph-mon[35013]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:31.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:31 smithi204 ceph-mon[35013]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:58:32.714 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T02:58:32.607+0000 7fd9c6e00640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T02:58:32.714 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T02:58:32.613+0000 7fd9c1e00640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:58:32.714 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-mon[30361]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:58:32.714 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-mon[30361]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:58:32.714 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-mon[30361]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:58:32.714 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:32.715 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:32 smithi057 ceph-mon[30361]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:58:32.892 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:32 smithi181 ceph-mon[35320]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:58:32.892 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:32 smithi181 ceph-mon[35320]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:32 smithi181 ceph-mon[35320]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:32 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:32 smithi181 ceph-mon[35320]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:32 smithi204 ceph-mon[35013]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:32 smithi204 ceph-mon[35013]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:32 smithi204 ceph-mon[35013]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:32 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:32 smithi204 ceph-mon[35013]: from='osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] boot 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:58:33.776 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] boot 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:58:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: osd.3 [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] boot 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:58:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:58:34.069 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:58:34.095 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_3 2024-09-12T02:58:34.345 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:58:34.788 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:34 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:58:34.788 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:34 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:58:34.788 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:34 smithi181 ceph-mon[35320]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/1043241696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec1031f0 msgr2=0x7ff2ec1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/1043241696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec1031f0 0x7ff2ec1035f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009900 tx=0x7ff2e802a410 comp rx=0 tx=0).stop 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/1043241696 shutdown_connections 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/1043241696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec100350 0x7ff2ec10a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/1043241696 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2ec103bc0 0x7ff2ec0ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/1043241696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec1031f0 0x7ff2ec1035f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:34.819 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/1043241696 >> 172.21.15.181:0/1043241696 conn(0x7ff2ec0fba40 msgr2=0x7ff2ec0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/1043241696 shutdown_connections 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.787+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/1043241696 wait complete. 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 Processor -- start 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 -- start start 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec1031f0 0x7ff2ec198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2ec103bc0 0x7ff2ec19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10d9a0 con 0x7ff2ec103bc0 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f1bc6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10d820 con 0x7ff2ec100350 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f3e2a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10db20 con 0x7ff2ec1031f0 2024-09-12T02:58:34.820 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f1bc6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:41056/0 (socket says 172.21.15.181:41056) 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.788+0000 7ff2f1bc6640 1 -- 172.21.15.181:0/2839321844 learned_addr learned my addr 172.21.15.181:0/2839321844 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 -- 172.21.15.181:0/2839321844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec1031f0 msgr2=0x7ff2ec198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec1031f0 0x7ff2ec198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 -- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2ec103bc0 msgr2=0x7ff2ec19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2ec103bc0 0x7ff2ec19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 -- 172.21.15.181:0/2839321844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec19fd70 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f1bc6640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005ab0 tx=0x7ff2e8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e803f030 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e8003ce0 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec1a0000 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8035070 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.789+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1a0560 con 0x7ff2ec100350 2024-09-12T02:58:34.821 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.790+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec109220 con 0x7ff2ec100350 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.791+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff2e8042030 con 0x7ff2ec100350 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.792+0000 7ff2e2ffd640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 0x7ff2d80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.792+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3698+0+0 (secure 0 0 0) 0x7ff2e80ee550 con 0x7ff2ec100350 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.792+0000 7ff2f13c5640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 0x7ff2d80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.792+0000 7ff2f13c5640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 0x7ff2d80a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec199660 tx=0x7ff2dc006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:34.822 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.796+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2e80bc6b0 con 0x7ff2ec100350 2024-09-12T02:58:34.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:34 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:58:34.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:34 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:58:34.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:34 smithi057 ceph-mon[30361]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:58:34.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:34 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:58:34.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:34 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:58:34.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:34 smithi204 ceph-mon[35013]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:58:34.971 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:34.971+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff2ec062630 con 0x7ff2d80a2bb0 2024-09-12T02:58:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 126 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:35.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:35.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:35.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:35 smithi057 ceph-mon[30361]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:58:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 126 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:35.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.896 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:35 smithi204 ceph-mon[35013]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:58:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 126 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:35.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:35.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:35 smithi181 ceph-mon[35320]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:58:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:36 smithi057 ceph-mon[30361]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:36 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi057 2024-09-12T02:58:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:36 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 8547M 2024-09-12T02:58:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:36 smithi057 ceph-mon[30361]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:58:36.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:36 smithi204 ceph-mon[35013]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:36.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:36 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi057 2024-09-12T02:58:36.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:36 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 8547M 2024-09-12T02:58:36.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:36 smithi204 ceph-mon[35013]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:58:36.920 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:36 smithi181 ceph-mon[35320]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:36.920 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:36 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi057 2024-09-12T02:58:36.920 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:36 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 8547M 2024-09-12T02:58:36.921 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:36 smithi181 ceph-mon[35320]: osdmap e27: 4 total, 4 up, 4 in 2024-09-12T02:58:37.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/254844390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]': finished 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:58:37.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:37 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:58:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:58:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/254844390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]': finished 2024-09-12T02:58:37.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:58:37.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:37 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:58:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: osdmap e28: 4 total, 4 up, 4 in 2024-09-12T02:58:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/254844390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]: dispatch 2024-09-12T02:58:37.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ed45fbb-6c70-460b-8f77-77f8810b2e0e"}]': finished 2024-09-12T02:58:37.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:58:37.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:37 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:58:38.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:38 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/3893646583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:38.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:38 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/3893646583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:39.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:38 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/3893646583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:58:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:39 smithi204 ceph-mon[35013]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:39 smithi181 ceph-mon[35320]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:40.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:39 smithi057 ceph-mon[30361]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:41 smithi204 ceph-mon[35013]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:41 smithi181 ceph-mon[35320]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:42.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:41 smithi057 ceph-mon[30361]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:43 smithi204 ceph-mon[35013]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:58:43.903 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:43 smithi181 ceph-mon[35320]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:58:44.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:43 smithi057 ceph-mon[30361]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:58:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:45 smithi204 ceph-mon[35013]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T02:58:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:45 smithi181 ceph-mon[35320]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T02:58:46.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:45 smithi057 ceph-mon[30361]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-12T02:58:46.691 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:58:46.691 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:47.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:58:47.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:47.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:58:47.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:47.915 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:47 smithi181 ceph-mon[35320]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T02:58:47.915 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:47 smithi181 ceph-mon[35320]: Deploying daemon osd.4 on smithi181 2024-09-12T02:58:48.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:47 smithi204 ceph-mon[35013]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T02:58:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:47 smithi204 ceph-mon[35013]: Deploying daemon osd.4 on smithi181 2024-09-12T02:58:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:47 smithi057 ceph-mon[30361]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T02:58:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:47 smithi057 ceph-mon[30361]: Deploying daemon osd.4 on smithi181 2024-09-12T02:58:49.918 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:49 smithi057 ceph-mon[30361]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T02:58:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:49 smithi181 ceph-mon[35320]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T02:58:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:49 smithi204 ceph-mon[35013]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-12T02:58:51.041 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:50 smithi181 ceph-mon[35320]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:51.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:50 smithi057 ceph-mon[30361]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:51.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:50 smithi204 ceph-mon[35013]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:53.066 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:52 smithi181 ceph-mon[35320]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:53.066 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:53.066 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:53.066 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:53.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:52 smithi057 ceph-mon[30361]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:53.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:53.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:53.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:53.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:52 smithi204 ceph-mon[35013]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:58:53.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:53.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:53.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:54.927 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:54 smithi181 ceph-mon[35320]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:55.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:54 smithi057 ceph-mon[30361]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:54 smithi204 ceph-mon[35013]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:56.372 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.371+0000 7ff2e2ffd640 1 -- 172.21.15.181:0/2839321844 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2ec062630 con 0x7ff2d80a2bb0 2024-09-12T02:58:56.373 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 4 on host 'smithi181' 2024-09-12T02:58:56.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 msgr2=0x7ff2d80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 0x7ff2d80a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec199660 tx=0x7ff2dc006c90 comp rx=0 tx=0).stop 2024-09-12T02:58:56.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 msgr2=0x7ff2ec198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.375 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005ab0 tx=0x7ff2e8004360 comp rx=0 tx=0).stop 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 shutdown_connections 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2d80a2bb0 0x7ff2d80a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2ec103bc0 0x7ff2ec19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2ec1031f0 0x7ff2ec198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 --2- 172.21.15.181:0/2839321844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2ec100350 0x7ff2ec198140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 >> 172.21.15.181:0/2839321844 conn(0x7ff2ec0fba40 msgr2=0x7ff2ec1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 shutdown_connections 2024-09-12T02:58:56.376 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:58:56.374+0000 7ff2f3e2a640 1 -- 172.21.15.181:0/2839321844 wait complete. 2024-09-12T02:58:57.042 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:56 smithi181 ceph-mon[35320]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:57.043 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:57.043 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.043 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.068 DEBUG:teuthology.orchestra.run.smithi181:osd.4> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T02:58:57.070 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-09-12T02:58:57.071 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:58:57.099 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:56 smithi204 ceph-mon[35013]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:57.099 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:57.099 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.099 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:56 smithi057 ceph-mon[30361]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:58:57.324 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:58:58.686 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 02:58:58 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T02:58:58.591+0000 7f721a191640 -1 osd.4 0 log_to_monitors true 2024-09-12T02:58:58.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:58 smithi181 ceph-mon[35320]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:58.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:58 smithi181 ceph-mon[35320]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:58.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:58 smithi181 ceph-mon[35320]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:59.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:58 smithi204 ceph-mon[35013]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:59.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:58 smithi204 ceph-mon[35013]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:59.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:58 smithi204 ceph-mon[35013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:58 smithi057 ceph-mon[30361]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:58:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:58 smithi057 ceph-mon[30361]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:58 smithi057 ceph-mon[30361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:58:59.803 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T02:58:59.830 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-09-12T02:59:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:59 smithi057 ceph-mon[30361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:59:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:59 smithi057 ceph-mon[30361]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:59 smithi057 ceph-mon[30361]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:59:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:58:59 smithi057 ceph-mon[30361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.121 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:59:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:59 smithi204 ceph-mon[35013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:59:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:59 smithi204 ceph-mon[35013]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:59 smithi204 ceph-mon[35013]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:59:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:58:59 smithi204 ceph-mon[35013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-mon[35320]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:59:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-mon[35320]: from='osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-mon[35320]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:59:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-mon[35320]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T02:59:00.204 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T02:58:59.824+0000 7f7216114640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T02:59:00.204 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 02:58:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T02:58:59.830+0000 7f7211f3c640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 -- 172.21.15.204:0/1454094707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64101d10 msgr2=0x7faa64102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 --2- 172.21.15.204:0/1454094707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64101d10 0x7faa64102190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009900 tx=0x7faa4c02a410 comp rx=0 tx=0).stop 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 -- 172.21.15.204:0/1454094707 shutdown_connections 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 --2- 172.21.15.204:0/1454094707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa64108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 --2- 172.21.15.204:0/1454094707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64101d10 0x7faa64102190 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 --2- 172.21.15.204:0/1454094707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64100b10 0x7faa64100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 -- 172.21.15.204:0/1454094707 >> 172.21.15.204:0/1454094707 conn(0x7faa640fc2a0 msgr2=0x7faa640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 -- 172.21.15.204:0/1454094707 shutdown_connections 2024-09-12T02:59:00.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.561+0000 7faa69cea640 1 -- 172.21.15.204:0/1454094707 wait complete. 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 Processor -- start 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 -- start start 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64101d10 0x7faa6419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa6419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410c810 con 0x7faa64101d10 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410c690 con 0x7faa641026d0 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa69cea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faa6410c990 con 0x7faa64100b10 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa637fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:60868/0 (socket says 172.21.15.204:60868) 2024-09-12T02:59:00.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa637fe640 1 -- 172.21.15.204:0/372056813 learned_addr learned my addr 172.21.15.204:0/372056813 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.562+0000 7faa63fff640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa6419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa62ffd640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64101d10 0x7faa6419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 -- 172.21.15.204:0/372056813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 msgr2=0x7faa6419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa6419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 -- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64101d10 msgr2=0x7faa6419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64101d10 0x7faa6419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 -- 172.21.15.204:0/372056813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6419fbc0 con 0x7faa64100b10 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa63fff640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa6419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa637fe640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b450 tx=0x7faa5400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54015020 con 0x7faa64100b10 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54011d70 con 0x7faa64100b10 2024-09-12T02:59:00.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6419feb0 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400f340 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.563+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a0330 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.564+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa64107f10 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.566+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa54018020 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.567+0000 7faa60ff9640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 0x7faa500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.567+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7faa540cdd90 con 0x7faa64100b10 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.567+0000 7faa62ffd640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 0x7faa500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.568+0000 7faa62ffd640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 0x7faa500a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0314e0 tx=0x7faa4c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.570+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa5401f020 con 0x7faa64100b10 2024-09-12T02:59:00.749 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:00.747+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faa64062630 con 0x7faa500a2b80 2024-09-12T02:59:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:59:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:59:01.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi181 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-12T02:59:01.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.122 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi181 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi181 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 8544M 2024-09-12T02:59:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:01.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:01.993 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:59:01.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:59:01.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:01.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] boot 2024-09-12T02:59:01.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:59:01.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:01 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] boot 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:59:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:01 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:59:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:59:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: osd.4 [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] boot 2024-09-12T02:59:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:59:02.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:01 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:59:03.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:02 smithi057 ceph-mon[30361]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:03.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:02 smithi057 ceph-mon[30361]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:59:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:02 smithi204 ceph-mon[35013]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:03.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:02 smithi204 ceph-mon[35013]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:59:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:02 smithi181 ceph-mon[35320]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:59:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:02 smithi181 ceph-mon[35320]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/606442141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]': finished 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:03 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/3156957287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/606442141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]': finished 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:03 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/3156957287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/606442141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]: dispatch 2024-09-12T02:59:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08fe7507-dac1-4941-b250-aa6352886ca7"}]': finished 2024-09-12T02:59:04.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:59:04.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:04.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:03 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/3156957287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:05.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:05 smithi057 ceph-mon[30361]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:05.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:05 smithi204 ceph-mon[35013]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:05.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:05 smithi181 ceph-mon[35320]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:07.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:07 smithi057 ceph-mon[30361]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:07.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:07 smithi204 ceph-mon[35013]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:07.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:07 smithi181 ceph-mon[35320]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:09.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:09 smithi204 ceph-mon[35013]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:09.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:09 smithi057 ceph-mon[30361]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:09.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:09 smithi181 ceph-mon[35320]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:11.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:11 smithi057 ceph-mon[30361]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:11.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:11 smithi204 ceph-mon[35013]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:11.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:11 smithi181 ceph-mon[35320]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:12.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:59:12.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:12.357 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:12 smithi204 ceph-mon[35013]: Deploying daemon osd.5 on smithi204 2024-09-12T02:59:12.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:59:12.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:12.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:12 smithi057 ceph-mon[30361]: Deploying daemon osd.5 on smithi204 2024-09-12T02:59:12.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:59:12.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:12.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:12 smithi181 ceph-mon[35320]: Deploying daemon osd.5 on smithi204 2024-09-12T02:59:13.277 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:13 smithi204 ceph-mon[35013]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:13 smithi057 ceph-mon[30361]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:13 smithi181 ceph-mon[35320]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:15.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:15 smithi204 ceph-mon[35013]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:15.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:15 smithi057 ceph-mon[30361]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:15.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:15 smithi181 ceph-mon[35320]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:17.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:17 smithi057 ceph-mon[30361]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:17.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:17 smithi204 ceph-mon[35013]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:17.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:17 smithi181 ceph-mon[35320]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:18.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:18.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:18.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:18.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:18.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:19.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:19 smithi204 ceph-mon[35013]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:19.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:19 smithi057 ceph-mon[30361]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:19.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:19 smithi181 ceph-mon[35320]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:21.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.840 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:21.840 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.840 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.840 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:21.841 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:21.841 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:21.889 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.888+0000 7faa60ff9640 1 -- 172.21.15.204:0/372056813 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faa64062630 con 0x7faa500a2b80 2024-09-12T02:59:21.889 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 msgr2=0x7faa500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 0x7faa500a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0314e0 tx=0x7faa4c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 msgr2=0x7faa6406b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b450 tx=0x7faa5400b910 comp rx=0 tx=0).stop 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 shutdown_connections 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7faa500a2b80 0x7faa500a5040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa641026d0 0x7faa6419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7faa64101d10 0x7faa6419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:21.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 --2- 172.21.15.204:0/372056813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa64100b10 0x7faa6406b330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:21.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 >> 172.21.15.204:0/372056813 conn(0x7faa640fc2a0 msgr2=0x7faa641050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:21.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 shutdown_connections 2024-09-12T02:59:21.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T02:59:21.890+0000 7faa69cea640 1 -- 172.21.15.204:0/372056813 wait complete. 2024-09-12T02:59:22.528 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T02:59:22.530 INFO:tasks.cephadm:Deploying osd.6 on smithi057 with /dev/vg_nvme/lv_2... 2024-09-12T02:59:22.530 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:59:22.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:22.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.726 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:22.726 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.727 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:22.909 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:23.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:23 smithi057 ceph-mon[30361]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:23 smithi181 ceph-mon[35320]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:23.767 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:23 smithi204 ceph-mon[35013]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:23.767 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 02:59:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T02:59:23.501+0000 7f177546e640 -1 osd.5 0 log_to_monitors true 2024-09-12T02:59:24.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:24 smithi057 ceph-mon[30361]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:24.665 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:24 smithi057 ceph-mon[30361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:24 smithi181 ceph-mon[35320]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:24 smithi181 ceph-mon[35320]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:24.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:24 smithi204 ceph-mon[35013]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:24.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:24 smithi204 ceph-mon[35013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:59:25.601 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T02:59:25.463+0000 7f17713f1640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T02:59:25.601 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T02:59:25.472+0000 7f176d219640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:59:25.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:25.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:59:25.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:59:25.601 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:25.602 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:25.602 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:25 smithi204 ceph-mon[35013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:25 smithi057 ceph-mon[30361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:59:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:59:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: from='osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:25 smithi181 ceph-mon[35320]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T02:59:26.525 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T02:59:26.558 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_2 2024-09-12T02:59:26.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:59:26.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:59:26.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi204 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] boot 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:59:26.609 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi204 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] boot 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:59:26.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.933 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi204 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: osd.5 [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] boot 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:59:26.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:59:27.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 -- 172.21.15.57:0/2030132398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8105810 msgr2=0x7f60b810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:27.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 --2- 172.21.15.57:0/2030132398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8105810 0x7f60b810c0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009ca0 tx=0x7f60b002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.532 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 -- 172.21.15.57:0/2030132398 shutdown_connections 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 --2- 172.21.15.57:0/2030132398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8105810 0x7f60b810c0a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 --2- 172.21.15.57:0/2030132398 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8104e50 0x7f60b81052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 --2- 172.21.15.57:0/2030132398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8103c50 0x7f60b8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 -- 172.21.15.57:0/2030132398 >> 172.21.15.57:0/2030132398 conn(0x7f60b80ff3e0 msgr2=0x7f60b8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 -- 172.21.15.57:0/2030132398 shutdown_connections 2024-09-12T02:59:27.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.531+0000 7f60bd200640 1 -- 172.21.15.57:0/2030132398 wait complete. 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.532+0000 7f60bd200640 1 Processor -- start 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.532+0000 7f60bd200640 1 -- start start 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 0x7f60b819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8105810 0x7f60b81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:27.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810f920 con 0x7f60b8103c50 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810f7a0 con 0x7f60b8104e50 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60bd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810faa0 con 0x7f60b8105810 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8105810 0x7f60b81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 0x7f60b819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54692/0 (socket says 172.21.15.57:54692) 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b7400640 1 -- 172.21.15.57:0/811119360 learned_addr learned my addr 172.21.15.57:0/811119360 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 -- 172.21.15.57:0/811119360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8105810 msgr2=0x7f60b81a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8105810 0x7f60b81a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 -- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 msgr2=0x7f60b8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b6a00640 1 -- 172.21.15.57:0/811119360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81a1b10 con 0x7f60b8104e50 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.533+0000 7f60b7400640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60b6a00640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 0x7f60b819a420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e9f0 tx=0x7f60a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a400cd80 con 0x7f60b8104e50 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a40035f0 con 0x7f60b8104e50 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4010690 con 0x7f60b8104e50 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b81a1e00 con 0x7f60b8104e50 2024-09-12T02:59:27.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.534+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b81a2260 con 0x7f60b8104e50 2024-09-12T02:59:27.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.535+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b810af80 con 0x7f60b8104e50 2024-09-12T02:59:27.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.538+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f60a4003b30 con 0x7f60b8104e50 2024-09-12T02:59:27.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.539+0000 7f60afe00640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 0x7f609c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:27.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.539+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7f60a40ce390 con 0x7f60b8104e50 2024-09-12T02:59:27.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.539+0000 7f60b7400640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 0x7f609c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:27.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.540+0000 7f60b7400640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 0x7f609c0a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60a80058c0 tx=0x7f60a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:27.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.541+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60a409c0f0 con 0x7f60b8104e50 2024-09-12T02:59:27.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:27.734+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f60b8062630 con 0x7f609c0a2c30 2024-09-12T02:59:27.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:27 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:59:27.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:27 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:59:27.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:27 smithi057 ceph-mon[30361]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:27.736 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:27 smithi057 ceph-mon[30361]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:59:27.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:27 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:59:27.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:27 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:59:27.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:27 smithi204 ceph-mon[35013]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:27.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:27 smithi204 ceph-mon[35013]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:59:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:27 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:59:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:27 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:59:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:27 smithi181 ceph-mon[35320]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:59:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:27 smithi181 ceph-mon[35320]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:59:28.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:28 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:28.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:28 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:28.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:28 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:28.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:28 smithi057 ceph-mon[30361]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:59:28.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:28 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:28.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:28 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:28.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:28 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:28.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:28 smithi204 ceph-mon[35013]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:59:28.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:28 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:28.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:28 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:28.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:28 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:28.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:28 smithi181 ceph-mon[35320]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:59:29.765 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:29 smithi057 ceph-mon[30361]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:29.765 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:29 smithi057 ceph-mon[30361]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:29 smithi204 ceph-mon[35013]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:29 smithi204 ceph-mon[35013]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:29 smithi181 ceph-mon[35320]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:29 smithi181 ceph-mon[35320]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:30.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:30 smithi057 ceph-mon[30361]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:59:30.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:30 smithi057 ceph-mon[30361]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:59:30.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:30 smithi204 ceph-mon[35013]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:59:30.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:30 smithi204 ceph-mon[35013]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:59:30.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:30 smithi181 ceph-mon[35320]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T02:59:30.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:30 smithi181 ceph-mon[35320]: osdmap e41: 6 total, 6 up, 6 in 2024-09-12T02:59:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:31 smithi204 ceph-mon[35013]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:31 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]: dispatch 2024-09-12T02:59:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:31 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]': finished 2024-09-12T02:59:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:31 smithi204 ceph-mon[35013]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:59:31.765 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:31 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:31 smithi057 ceph-mon[30361]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:31 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]: dispatch 2024-09-12T02:59:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:31 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]': finished 2024-09-12T02:59:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:31 smithi057 ceph-mon[30361]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:59:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:31 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:31 smithi181 ceph-mon[35320]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:31 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]: dispatch 2024-09-12T02:59:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:31 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2937441614' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "791b2b8c-0c86-49ca-a977-e6cb195ad8ae"}]': finished 2024-09-12T02:59:31.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:31 smithi181 ceph-mon[35320]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:59:31.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:31 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:32.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:32 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/782165099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:32.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:32 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/782165099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:32.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:32 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/782165099' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:59:33.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:33 smithi057 ceph-mon[30361]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:33 smithi204 ceph-mon[35013]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:33.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:33 smithi181 ceph-mon[35320]: pgmap v148: 1 pgs: 1 peering; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:59:35.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:35 smithi057 ceph-mon[30361]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T02:59:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:35 smithi204 ceph-mon[35013]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T02:59:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:35 smithi181 ceph-mon[35320]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-12T02:59:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:37 smithi057 ceph-mon[30361]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T02:59:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:37 smithi204 ceph-mon[35013]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T02:59:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:37 smithi181 ceph-mon[35320]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-12T02:59:39.833 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:39 smithi057 ceph-mon[30361]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:59:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:39 smithi204 ceph-mon[35013]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:59:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:39 smithi181 ceph-mon[35320]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:59:40.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:59:40.789 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:40.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:59:40.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:59:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:41 smithi057 ceph-mon[30361]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:41 smithi057 ceph-mon[30361]: Deploying daemon osd.6 on smithi057 2024-09-12T02:59:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:41 smithi204 ceph-mon[35013]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:41 smithi204 ceph-mon[35013]: Deploying daemon osd.6 on smithi057 2024-09-12T02:59:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:41 smithi181 ceph-mon[35320]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:41 smithi181 ceph-mon[35320]: Deploying daemon osd.6 on smithi057 2024-09-12T02:59:43.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:43 smithi057 ceph-mon[30361]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:43 smithi204 ceph-mon[35013]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:43 smithi181 ceph-mon[35320]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:45.820 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:45 smithi057 ceph-mon[30361]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:45 smithi204 ceph-mon[35013]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:45 smithi181 ceph-mon[35320]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:47.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:47 smithi057 ceph-mon[30361]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:47.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:47 smithi204 ceph-mon[35013]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:47 smithi181 ceph-mon[35320]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:48.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:48.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:48.585 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:48.808 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:48.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:48.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:49.851 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:49 smithi057 ceph-mon[30361]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:49.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:49 smithi204 ceph-mon[35013]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:49 smithi181 ceph-mon[35320]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:51.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:51.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.410+0000 7f60afe00640 1 -- 172.21.15.57:0/811119360 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60b8062630 con 0x7f609c0a2c30 2024-09-12T02:59:52.412 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 6 on host 'smithi057' 2024-09-12T02:59:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 msgr2=0x7f609c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 0x7f609c0a50f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f60a80058c0 tx=0x7f60a8005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 msgr2=0x7f60b819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 0x7f60b819a420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e9f0 tx=0x7f60a400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 shutdown_connections 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f609c0a2c30 0x7f609c0a50f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60b8105810 0x7f60b81a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b8104e50 0x7f60b819a420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 --2- 172.21.15.57:0/811119360 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60b8103c50 0x7f60b8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 >> 172.21.15.57:0/811119360 conn(0x7f60b80ff3e0 msgr2=0x7f60b8108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 shutdown_connections 2024-09-12T02:59:52.414 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T02:59:52.413+0000 7f60bd200640 1 -- 172.21.15.57:0/811119360 wait complete. 2024-09-12T02:59:52.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:52.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:52.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:52.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:52.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:53.198 DEBUG:teuthology.orchestra.run.smithi057:osd.6> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T02:59:53.201 INFO:tasks.cephadm:Deploying osd.7 on smithi181 with /dev/vg_nvme/lv_2... 2024-09-12T02:59:53.202 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:59:53.440 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:59:53.497 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 02:59:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T02:59:53.406+0000 7fedc3048640 -1 osd.6 0 log_to_monitors true 2024-09-12T02:59:53.762 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:53 smithi057 ceph-mon[30361]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:53.762 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:53 smithi057 ceph-mon[30361]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:59:53.839 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:53 smithi181 ceph-mon[35320]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:53.839 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:53 smithi181 ceph-mon[35320]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:59:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:53 smithi204 ceph-mon[35013]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:53 smithi204 ceph-mon[35013]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:59:54.577 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-mon[30361]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:59:54.577 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-mon[30361]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:59:54.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:54 smithi181 ceph-mon[35320]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:59:54.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:54 smithi181 ceph-mon[35320]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:59:54.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:54 smithi181 ceph-mon[35320]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:59:54.859 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:54 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:54.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-mon[30361]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:59:54.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:54.870 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T02:59:54.592+0000 7fedbdc00640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T02:59:54.870 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 02:59:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T02:59:54.602+0000 7fedb8c00640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:59:54.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:54 smithi204 ceph-mon[35013]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:59:54.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:54 smithi204 ceph-mon[35013]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:59:54.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:54 smithi204 ceph-mon[35013]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T02:59:54.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:54 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.818 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:55 smithi181 ceph-mon[35320]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:55.818 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:55 smithi181 ceph-mon[35320]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:59:55.818 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:55 smithi181 ceph-mon[35320]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:59:55.818 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.818 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:55 smithi057 ceph-mon[30361]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:55 smithi057 ceph-mon[30361]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:59:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:55 smithi057 ceph-mon[30361]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:59:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:55 smithi204 ceph-mon[35013]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:55 smithi204 ceph-mon[35013]: from='osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T02:59:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:55 smithi204 ceph-mon[35013]: osdmap e44: 7 total, 6 up, 7 in 2024-09-12T02:59:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.087 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T02:59:56.115 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_2 2024-09-12T02:59:56.359 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:59:56.746 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T02:59:56.746 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T02:59:56.746 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.746 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] boot 2024-09-12T02:59:56.746 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:59:56.747 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 -- 172.21.15.181:0/2649585088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488102720 msgr2=0x7f8488104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 --2- 172.21.15.181:0/2649585088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488102720 0x7f8488104b10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8478009ca0 tx=0x7f847802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 -- 172.21.15.181:0/2649585088 shutdown_connections 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 --2- 172.21.15.181:0/2649585088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488107980 0x7f8488109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 --2- 172.21.15.181:0/2649585088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 0x7f8488107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 --2- 172.21.15.181:0/2649585088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488102720 0x7f8488104b10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 -- 172.21.15.181:0/2649585088 >> 172.21.15.181:0/2649585088 conn(0x7f84880fc280 msgr2=0x7f84880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 -- 172.21.15.181:0/2649585088 shutdown_connections 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.829+0000 7f848f36b640 1 -- 172.21.15.181:0/2649585088 wait complete. 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 Processor -- start 2024-09-12T02:59:56.855 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 -- start start 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488102720 0x7f8488197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 0x7f84881984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f848810d890 con 0x7f8488102720 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f848810d710 con 0x7f8488107980 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848f36b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f848810da10 con 0x7f8488105050 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:53112/0 (socket says 172.21.15.181:53112) 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 -- 172.21.15.181:0/1647797776 learned_addr learned my addr 172.21.15.181:0/1647797776 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848c906640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 0x7f84881984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d107640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488102720 0x7f8488197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:56.856 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 -- 172.21.15.181:0/1647797776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 msgr2=0x7f84881984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 0x7f84881984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 -- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488102720 msgr2=0x7f8488197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488102720 0x7f8488197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 -- 172.21.15.181:0/1647797776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848819fba0 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.830+0000 7f848d908640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f848000d750 tx=0x7f848000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.831+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480017070 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.831+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8480005500 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.831+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480004d90 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.831+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848819fe90 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.831+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84881a02f0 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.832+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84880fdd70 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.833+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8480004f30 con 0x7f8488107980 2024-09-12T02:59:56.857 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.834+0000 7f84867fc640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 0x7f84700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:56.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.834+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f84800cdf90 con 0x7f8488107980 2024-09-12T02:59:56.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.834+0000 7f848d107640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 0x7f84700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:56.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.835+0000 7f848d107640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 0x7f84700a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8478006010 tx=0x7f8478000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:56.858 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:56.837+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f848009b630 con 0x7f8488107980 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] boot 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:59:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T02:59:56.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T02:59:56.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:56.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: osd.6 [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] boot 2024-09-12T02:59:56.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:59:56.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:59:57.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T02:59:57.014+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8488062630 con 0x7f84700a2c30 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:57.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:57.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:57.676 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: osdmap e46: 7 total, 7 up, 7 in 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:59:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:58.612 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:58 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi057 2024-09-12T02:59:58.613 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:58 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 5698M 2024-09-12T02:59:58.613 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:58 smithi181 ceph-mon[35320]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:58.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:58 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi057 2024-09-12T02:59:58.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:58 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 5698M 2024-09-12T02:59:58.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:58 smithi057 ceph-mon[30361]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:58 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi057 2024-09-12T02:59:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:58 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 5698M 2024-09-12T02:59:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:58 smithi204 ceph-mon[35013]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/765324696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]': finished 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:59:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 02:59:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:59:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:59:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/765324696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]': finished 2024-09-12T02:59:59.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:59:59.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 02:59:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/765324696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]: dispatch 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45285c14-5770-46e0-8eb8-8f8e46d74882"}]': finished 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:59:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 02:59:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:00.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:00 smithi057 ceph-mon[30361]: overall HEALTH_OK 2024-09-12T03:00:00.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:00 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/2856345396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:00.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:00 smithi204 ceph-mon[35013]: overall HEALTH_OK 2024-09-12T03:00:00.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:00 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/2856345396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:00.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:00 smithi181 ceph-mon[35320]: overall HEALTH_OK 2024-09-12T03:00:00.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:00 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/2856345396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:01.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:01 smithi057 ceph-mon[30361]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:01 smithi204 ceph-mon[35013]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:01.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:01 smithi181 ceph-mon[35320]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:03 smithi057 ceph-mon[30361]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:03 smithi204 ceph-mon[35013]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:03 smithi181 ceph-mon[35320]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:05.890 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:05 smithi181 ceph-mon[35320]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:05 smithi204 ceph-mon[35013]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:06.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:05 smithi057 ceph-mon[30361]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:07 smithi181 ceph-mon[35320]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:07 smithi204 ceph-mon[35013]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:08.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:07 smithi057 ceph-mon[30361]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:08.889 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:08 smithi181 ceph-mon[35320]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:08.889 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:00:08.889 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:08 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:08 smithi204 ceph-mon[35013]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:00:08.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:08 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:08 smithi057 ceph-mon[30361]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:00:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:08 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:09 smithi204 ceph-mon[35013]: Deploying daemon osd.7 on smithi181 2024-09-12T03:00:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:09 smithi181 ceph-mon[35320]: Deploying daemon osd.7 on smithi181 2024-09-12T03:00:10.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:09 smithi057 ceph-mon[30361]: Deploying daemon osd.7 on smithi181 2024-09-12T03:00:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:11 smithi181 ceph-mon[35320]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:12.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:11 smithi057 ceph-mon[30361]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:11 smithi204 ceph-mon[35013]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:13.822 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:13 smithi181 ceph-mon[35320]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:13 smithi057 ceph-mon[30361]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:13 smithi204 ceph-mon[35013]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:14.925 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:14.925 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:14.925 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:15.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:15.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:15.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:15.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:15.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:15.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:15 smithi181 ceph-mon[35320]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:16.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:15 smithi057 ceph-mon[30361]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:15 smithi204 ceph-mon[35013]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:18.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:18.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:18.426 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.425+0000 7f84867fc640 1 -- 172.21.15.181:0/1647797776 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8488062630 con 0x7f84700a2c30 2024-09-12T03:00:18.426 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 7 on host 'smithi181' 2024-09-12T03:00:18.428 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 msgr2=0x7f84700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 0x7f84700a50f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8478006010 tx=0x7f8478000f40 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 msgr2=0x7f848819f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f848000d750 tx=0x7f848000dc10 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 shutdown_connections 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f84700a2c30 0x7f84700a50f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8488107980 0x7f848819f530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8488105050 0x7f84881984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 --2- 172.21.15.181:0/1647797776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8488102720 0x7f8488197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 >> 172.21.15.181:0/1647797776 conn(0x7f84880fc280 msgr2=0x7f8488103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 shutdown_connections 2024-09-12T03:00:18.429 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:00:18.428+0000 7f848f36b640 1 -- 172.21.15.181:0/1647797776 wait complete. 2024-09-12T03:00:19.137 DEBUG:teuthology.orchestra.run.smithi181:osd.7> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T03:00:19.140 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-09-12T03:00:19.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:00:19.398 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:00:19.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:19 smithi204 ceph-mon[35013]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:19.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:19.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:19.519 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:19.658 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:19 smithi181 ceph-mon[35320]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:19.658 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:19.658 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:19.658 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:19 smithi057 ceph-mon[30361]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:20.627 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:00:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:00:20.501+0000 7f7fb9b3a640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:00:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:21 smithi204 ceph-mon[35013]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:21 smithi204 ceph-mon[35013]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:21 smithi204 ceph-mon[35013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.686 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:21 smithi181 ceph-mon[35320]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:21.686 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:21 smithi181 ceph-mon[35320]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.686 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:21 smithi181 ceph-mon[35320]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:21 smithi057 ceph-mon[30361]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:21 smithi057 ceph-mon[30361]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:21 smithi057 ceph-mon[30361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:00:21.943 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T03:00:21.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-09-12T03:00:22.225 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:00:22.400+0000 7f7fb5abd640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:00:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:00:22.410+0000 7f7fb18e5640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- 172.21.15.204:0/3115047506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c102690 msgr2=0x7fe61c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 --2- 172.21.15.204:0/3115047506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c102690 0x7fe61c108f20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe614009ca0 tx=0x7fe61402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- 172.21.15.204:0/3115047506 shutdown_connections 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 --2- 172.21.15.204:0/3115047506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c102690 0x7fe61c108f20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 --2- 172.21.15.204:0/3115047506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c101cd0 0x7fe61c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 --2- 172.21.15.204:0/3115047506 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 0x7fe61c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- 172.21.15.204:0/3115047506 >> 172.21.15.204:0/3115047506 conn(0x7fe61c0fc280 msgr2=0x7fe61c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- 172.21.15.204:0/3115047506 shutdown_connections 2024-09-12T03:00:22.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- 172.21.15.204:0/3115047506 wait complete. 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 Processor -- start 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.728+0000 7fe6234fe640 1 -- start start 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 0x7fe61c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c102690 0x7fe61c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c10d8e0 con 0x7fe61c100ad0 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c10d760 con 0x7fe61c102690 2024-09-12T03:00:22.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6234fe640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c10da60 con 0x7fe61c101cd0 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:35070/0 (socket says 172.21.15.204:35070) 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 -- 172.21.15.204:0/337474345 learned_addr learned my addr 172.21.15.204:0/337474345 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe62129a640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 0x7fe61c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe621a9b640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c102690 0x7fe61c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 -- 172.21.15.204:0/337474345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c102690 msgr2=0x7fe61c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c102690 0x7fe61c19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 -- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 msgr2=0x7fe61c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 0x7fe61c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 -- 172.21.15.204:0/337474345 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c19fb60 con 0x7fe61c101cd0 2024-09-12T03:00:22.756 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe620a99640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9f0 tx=0x7fe60c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.729+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00cd80 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.730+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c19fe50 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.730+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60c010910 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.730+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0188f0 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.730+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a0380 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.731+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c107e70 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.732+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe60c010ac0 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.732+0000 7fe6127fc640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 0x7fe6080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.733+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7fe60c0cebe0 con 0x7fe61c101cd0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.733+0000 7fe62129a640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 0x7fe6080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.733+0000 7fe62129a640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 0x7fe6080a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe61c101b30 tx=0x7fe618008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.735+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe60c09c7c0 con 0x7fe61c101cd0 2024-09-12T03:00:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:00:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T03:00:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: osdmap e49: 8 total, 7 up, 8 in 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:22.913 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:22.913+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe61c062630 con 0x7fe6080a2bb0 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi181 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:23.458 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:23.459 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] boot 2024-09-12T03:00:23.459 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:00:23.459 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:23 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi181 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] boot 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:00:23.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:23 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi181 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 5696M 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: osd.7 [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] boot 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:00:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:23 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:00:24.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:24 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T03:00:24.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:24 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T03:00:24.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:24 smithi204 ceph-mon[35013]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:24.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:24 smithi204 ceph-mon[35013]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:00:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:24 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T03:00:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:24 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T03:00:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:24 smithi181 ceph-mon[35320]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:24 smithi181 ceph-mon[35320]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:00:24.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:24 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T03:00:24.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:24 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T03:00:24.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:24 smithi057 ceph-mon[30361]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:24.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:24 smithi057 ceph-mon[30361]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/1248272195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]': finished 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:00:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/1248272195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]': finished 2024-09-12T03:00:25.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:00:25.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:25.801 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:25.801 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/1248272195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.802 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]: dispatch 2024-09-12T03:00:25.802 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2012481a-b076-4538-a4f4-09c1e17f89f4"}]': finished 2024-09-12T03:00:25.802 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:00:25.802 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:26 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/4007852343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:26 smithi181 ceph-mon[35320]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:00:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:26 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:26.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:26 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/4007852343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:26.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:26 smithi057 ceph-mon[30361]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:00:26.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:26 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:26.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:26 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/4007852343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:26.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:26 smithi204 ceph-mon[35013]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:00:26.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:26 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:27.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:27 smithi057 ceph-mon[30361]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:27.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:27 smithi057 ceph-mon[30361]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:00:27.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:27 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:27 smithi181 ceph-mon[35320]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:27 smithi181 ceph-mon[35320]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:00:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:27 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:27 smithi204 ceph-mon[35013]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:27 smithi204 ceph-mon[35013]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T03:00:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:27 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:29 smithi181 ceph-mon[35320]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:29 smithi057 ceph-mon[30361]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:29 smithi204 ceph-mon[35013]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:31.691 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:31 smithi204 ceph-mon[35013]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:31 smithi181 ceph-mon[35320]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:31 smithi057 ceph-mon[30361]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:33 smithi181 ceph-mon[35320]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:33.722 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:33 smithi204 ceph-mon[35013]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:33.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:33 smithi057 ceph-mon[30361]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:34.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:00:34.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:34 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:34.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:00:34.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:34 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:34.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:00:34.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:34 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:35 smithi181 ceph-mon[35320]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:35 smithi181 ceph-mon[35320]: Deploying daemon osd.8 on smithi204 2024-09-12T03:00:35.733 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:35 smithi204 ceph-mon[35013]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:35.733 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:35 smithi204 ceph-mon[35013]: Deploying daemon osd.8 on smithi204 2024-09-12T03:00:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:35 smithi057 ceph-mon[30361]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:35 smithi057 ceph-mon[30361]: Deploying daemon osd.8 on smithi204 2024-09-12T03:00:37.585 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:37 smithi204 ceph-mon[35013]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:37.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:37 smithi057 ceph-mon[30361]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:37 smithi181 ceph-mon[35320]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:39.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:39 smithi204 ceph-mon[35013]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:39 smithi057 ceph-mon[30361]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:39 smithi181 ceph-mon[35320]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:40.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:40.596 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:40.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:40.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:41.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:41 smithi204 ceph-mon[35013]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:41 smithi057 ceph-mon[30361]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:41 smithi181 ceph-mon[35320]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:43.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.635+0000 7fe6127fc640 1 -- 172.21.15.204:0/337474345 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe61c062630 con 0x7fe6080a2bb0 2024-09-12T03:00:43.636 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 msgr2=0x7fe6080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 0x7fe6080a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe61c101b30 tx=0x7fe618008040 comp rx=0 tx=0).stop 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 msgr2=0x7fe61c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00e9f0 tx=0x7fe60c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 shutdown_connections 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6080a2bb0 0x7fe6080a5070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c102690 0x7fe61c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.638 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe61c101cd0 0x7fe61c198470 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 --2- 172.21.15.204:0/337474345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe61c100ad0 0x7fe61c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.637+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 >> 172.21.15.204:0/337474345 conn(0x7fe61c0fc280 msgr2=0x7fe61c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:43.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.638+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 shutdown_connections 2024-09-12T03:00:43.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:00:43.638+0000 7fe6234fe640 1 -- 172.21.15.204:0/337474345 wait complete. 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:43 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:43.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:43 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:43.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:43 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.330 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T03:00:44.333 INFO:tasks.cephadm:Deploying osd.9 on smithi057 with /dev/vg_nvme/lv_1... 2024-09-12T03:00:44.333 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:00:44.718 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:44.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:44.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:45.836 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:45 smithi204 ceph-mon[35013]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:45.906 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:45 smithi057 ceph-mon[30361]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:45 smithi181 ceph-mon[35320]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:46.573 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:00:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:00:46.193+0000 7fc8d3df5640 -1 osd.8 0 log_to_monitors true 2024-09-12T03:00:46.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:46 smithi204 ceph-mon[35013]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:46.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:46 smithi204 ceph-mon[35013]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:46 smithi181 ceph-mon[35320]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:46 smithi181 ceph-mon[35320]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:47.068 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:46 smithi057 ceph-mon[30361]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:47.068 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:46 smithi057 ceph-mon[30361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:47.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:47.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:47.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:47.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:47.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:48.113 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:00:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:00:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:48.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:00:48.145 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi057:vg_nvme/lv_1 2024-09-12T03:00:48.536 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:48.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:00:48.311+0000 7fc8cfd78640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T03:00:48.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:00:48.321+0000 7fc8c7b98640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:00:48.920 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi204 2024-09-12T03:00:48.920 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-12T03:00:48.920 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:00:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T03:00:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:48.921 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:48 smithi057 ceph-mon[30361]: from='osd.8 ' entity='osd.8' 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi204 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:48.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:48 smithi181 ceph-mon[35320]: from='osd.8 ' entity='osd.8' 2024-09-12T03:00:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 -- 172.21.15.57:0/85877964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4103c50 msgr2=0x7fdaf4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/85877964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4103c50 0x7fdaf4104050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a410 comp rx=0 tx=0).stop 2024-09-12T03:00:49.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 -- 172.21.15.57:0/85877964 shutdown_connections 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/85877964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4105810 0x7fdaf410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/85877964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 0x7fdaf41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/85877964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4103c50 0x7fdaf4104050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 -- 172.21.15.57:0/85877964 >> 172.21.15.57:0/85877964 conn(0x7fdaf40ff3e0 msgr2=0x7fdaf4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 -- 172.21.15.57:0/85877964 shutdown_connections 2024-09-12T03:00:49.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.141+0000 7fdaf9c00640 1 -- 172.21.15.57:0/85877964 wait complete. 2024-09-12T03:00:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.142+0000 7fdaf9c00640 1 Processor -- start 2024-09-12T03:00:49.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.142+0000 7fdaf9c00640 1 -- start start 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 0x7fdaf4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 0x7fdaf419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf410f900 con 0x7fdaf4105810 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf410f780 con 0x7fdaf4103c50 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf410fa80 con 0x7fdaf4104e50 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 0x7fdaf419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52318/0 (socket says 172.21.15.57:52318) 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 0x7fdaf4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 -- 172.21.15.57:0/2365711458 learned_addr learned my addr 172.21.15.57:0/2365711458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:00:49.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 -- 172.21.15.57:0/2365711458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 msgr2=0x7fdaf419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 0x7fdaf419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 -- 172.21.15.57:0/2365711458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 msgr2=0x7fdaf4199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 0x7fdaf4199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.143+0000 7fdaf9200640 1 -- 172.21.15.57:0/2365711458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf41a1b20 con 0x7fdaf4105810 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf3e00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 0x7fdaf4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf9200640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00eb10 tx=0x7fdaec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:49.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004b20 con 0x7fdaf4105810 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi204 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: osdmap e56: 9 total, 8 up, 9 in 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:48 smithi204 ceph-mon[35013]: from='osd.8 ' entity='osd.8' 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec004cc0 con 0x7fdaf4105810 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf41a1e10 con 0x7fdaf4105810 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec010760 con 0x7fdaf4105810 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.144+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a2270 con 0x7fdaf4105810 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.145+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf410afa0 con 0x7fdaf4105810 2024-09-12T03:00:49.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.147+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdaec004fe0 con 0x7fdaf4105810 2024-09-12T03:00:49.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.149+0000 7fdaf0c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 0x7fdad80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:49.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.149+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7fdaec0cecb0 con 0x7fdaf4105810 2024-09-12T03:00:49.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.149+0000 7fdaf3e00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 0x7fdad80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:49.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.150+0000 7fdaf3e00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 0x7fdad80a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdae402aa00 tx=0x7fdae40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:49.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.154+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdaec09c500 con 0x7fdaf4105810 2024-09-12T03:00:49.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:00:49.347+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdaf4062630 con 0x7fdad80a2bb0 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] boot 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:00:49.947 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:49.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:49.948 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:49.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] boot 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:49.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: osd.8 [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] boot 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:00:50.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:50.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:50 smithi181 ceph-mon[35320]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:50.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:50 smithi181 ceph-mon[35320]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:00:51.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:50 smithi057 ceph-mon[30361]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:51.008 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:50 smithi057 ceph-mon[30361]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:00:51.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:50 smithi204 ceph-mon[35013]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi057:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:00:51.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:50 smithi204 ceph-mon[35013]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T03:00:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:51 smithi181 ceph-mon[35320]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:51 smithi181 ceph-mon[35320]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:00:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:51 smithi181 ceph-mon[35320]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:00:51.964 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:51 smithi057 ceph-mon[30361]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:51.964 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:51 smithi057 ceph-mon[30361]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:00:51.964 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:51 smithi057 ceph-mon[30361]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:00:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:51 smithi204 ceph-mon[35013]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:51 smithi204 ceph-mon[35013]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T03:00:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:51 smithi204 ceph-mon[35013]: osdmap e60: 9 total, 9 up, 9 in 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: pgmap v206: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]: dispatch 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]': finished 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:00:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:53 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/392932728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: pgmap v206: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T03:00:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]: dispatch 2024-09-12T03:00:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]': finished 2024-09-12T03:00:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:00:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:00:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:53 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/392932728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: pgmap v206: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:00:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: osdmap e61: 9 total, 9 up, 9 in 2024-09-12T03:00:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]: dispatch 2024-09-12T03:00:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2429944914' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51"}]': finished 2024-09-12T03:00:53.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T03:00:53.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:00:53.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:53 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/392932728' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:00:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:55 smithi057 ceph-mon[30361]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:00:55.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:55 smithi204 ceph-mon[35013]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:00:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:55 smithi181 ceph-mon[35320]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:00:57.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:57 smithi057 ceph-mon[30361]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:00:57.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:57 smithi204 ceph-mon[35013]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:00:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:57 smithi181 ceph-mon[35320]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:00:59.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:00:59 smithi057 ceph-mon[30361]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T03:00:59.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:00:59 smithi204 ceph-mon[35013]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T03:00:59.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:00:59 smithi181 ceph-mon[35320]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-12T03:01:01.521 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:01 smithi057 ceph-mon[30361]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:01:01.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:01 smithi204 ceph-mon[35013]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:01:01.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:01 smithi181 ceph-mon[35320]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:01:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:03 smithi057 ceph-mon[30361]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T03:01:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:01:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:03 smithi057 ceph-mon[30361]: Deploying daemon osd.9 on smithi057 2024-09-12T03:01:03.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:03 smithi204 ceph-mon[35013]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T03:01:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:01:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:03 smithi204 ceph-mon[35013]: Deploying daemon osd.9 on smithi057 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:03 smithi181 ceph-mon[35320]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:03 smithi181 ceph-mon[35320]: Deploying daemon osd.9 on smithi057 2024-09-12T03:01:05.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:05 smithi057 ceph-mon[30361]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:01:05.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:05 smithi204 ceph-mon[35013]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:01:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:05 smithi181 ceph-mon[35320]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-12T03:01:07.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:07 smithi057 ceph-mon[30361]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:07.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:07 smithi204 ceph-mon[35013]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:07 smithi181 ceph-mon[35320]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:09.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:09 smithi057 ceph-mon[30361]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:09 smithi204 ceph-mon[35013]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:09.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:09 smithi181 ceph-mon[35320]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:10.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:10.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:10.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:11 smithi057 ceph-mon[30361]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:11 smithi204 ceph-mon[35013]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:11 smithi181 ceph-mon[35320]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:13.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:13.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.156+0000 7fdaf0c00640 1 -- 172.21.15.57:0/2365711458 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaf4062630 con 0x7fdad80a2bb0 2024-09-12T03:01:14.157 INFO:teuthology.orchestra.run.smithi057.stdout:Created osd(s) 9 on host 'smithi057' 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 msgr2=0x7fdad80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 0x7fdad80a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdae402aa00 tx=0x7fdae40315b0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 msgr2=0x7fdaf41a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00eb10 tx=0x7fdaec00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 shutdown_connections 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdad80a2bb0 0x7fdad80a5070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdaf4105810 0x7fdaf41a14b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdaf4104e50 0x7fdaf419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 --2- 172.21.15.57:0/2365711458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdaf4103c50 0x7fdaf4199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:14.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.158+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 >> 172.21.15.57:0/2365711458 conn(0x7fdaf40ff3e0 msgr2=0x7fdaf4108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:14.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.159+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 shutdown_connections 2024-09-12T03:01:14.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:01:14.159+0000 7fdaf9c00640 1 -- 172.21.15.57:0/2365711458 wait complete. 2024-09-12T03:01:14.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:14.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:14 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:14.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:14 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:14.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:14 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:14.827 DEBUG:teuthology.orchestra.run.smithi057:osd.9> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T03:01:14.831 INFO:tasks.cephadm:Deploying osd.10 on smithi181 with /dev/vg_nvme/lv_1... 2024-09-12T03:01:14.832 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:01:15.087 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:01:15.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:15 smithi057 ceph-mon[30361]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:15.469 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:15 smithi181 ceph-mon[35320]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:15 smithi204 ceph-mon[35013]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:16.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:01:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:01:15.751+0000 7f3fee01d640 -1 osd.9 0 log_to_monitors true 2024-09-12T03:01:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:16 smithi057 ceph-mon[30361]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:16.424 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:16 smithi057 ceph-mon[30361]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:16.639 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:16 smithi181 ceph-mon[35320]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:16.639 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:16 smithi181 ceph-mon[35320]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:16.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:16 smithi204 ceph-mon[35013]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:16.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:16 smithi204 ceph-mon[35013]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:17.563 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:17 smithi181 ceph-mon[35320]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-mon[30361]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:01:17.382+0000 7f3fe8c00640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T03:01:17.621 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:01:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:01:17.396+0000 7f3fe3c00640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:01:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:01:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: from='osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: osdmap e63: 10 total, 9 up, 10 in 2024-09-12T03:01:17.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:17.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:17 smithi204 ceph-mon[35013]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]: dispatch 2024-09-12T03:01:17.994 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-12T03:01:18.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi181:vg_nvme/lv_1 2024-09-12T03:01:18.275 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:01:18.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:18 smithi057 ceph-mon[30361]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T03:01:18.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:18 smithi057 ceph-mon[30361]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T03:01:18.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:18 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:18 smithi204 ceph-mon[35013]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T03:01:18.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:18 smithi204 ceph-mon[35013]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T03:01:18.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:18 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.670 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:18 smithi181 ceph-mon[35320]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi057", "root=default"]}]': finished 2024-09-12T03:01:18.670 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:18 smithi181 ceph-mon[35320]: osdmap e64: 10 total, 9 up, 10 in 2024-09-12T03:01:18.670 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.670 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:18 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- 172.21.15.181:0/3733772815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490101c00 msgr2=0x7f2490102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- 172.21.15.181:0/3733772815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490101c00 0x7f2490102000 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2480009ca0 tx=0x7f248002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- 172.21.15.181:0/3733772815 shutdown_connections 2024-09-12T03:01:18.778 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- 172.21.15.181:0/3733772815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24901037c0 0x7f249010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- 172.21.15.181:0/3733772815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2490102e00 0x7f2490103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- 172.21.15.181:0/3733772815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490101c00 0x7f2490102000 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- 172.21.15.181:0/3733772815 >> 172.21.15.181:0/3733772815 conn(0x7f24900fd390 msgr2=0x7f24900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- 172.21.15.181:0/3733772815 shutdown_connections 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- 172.21.15.181:0/3733772815 wait complete. 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 Processor -- start 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- start start 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2490101c00 0x7f2490197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24901037c0 0x7f249019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f249010d8e0 con 0x7f2490101c00 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f249010d760 con 0x7f2490102e00 2024-09-12T03:01:18.779 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.753+0000 7f2496de9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f249010da60 con 0x7f24901037c0 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:33066/0 (socket says 172.21.15.181:33066) 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 -- 172.21.15.181:0/1844065476 learned_addr learned my addr 172.21.15.181:0/1844065476 (peer_addr_for_me v2:172.21.15.181:0/0) 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f2494b85640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2490101c00 0x7f2490197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f2495386640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24901037c0 0x7f249019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 -- 172.21.15.181:0/1844065476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24901037c0 msgr2=0x7f249019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24901037c0 0x7f249019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 -- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2490101c00 msgr2=0x7f2490197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2490101c00 0x7f2490197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 -- 172.21.15.181:0/1844065476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249019fb80 con 0x7f2490102e00 2024-09-12T03:01:18.780 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248ffff640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f247800b4a0 tx=0x7f247800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478014070 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2478010af0 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478017d10 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.754+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249019fe70 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.755+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24901a0350 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.756+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24901090a0 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.757+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2478017070 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.757+0000 7f248dffb640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 0x7f247c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.758+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6261+0+0 (secure 0 0 0) 0x7f24780cdbd0 con 0x7f2490102e00 2024-09-12T03:01:18.781 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.758+0000 7f2494b85640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 0x7f247c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.758+0000 7f2494b85640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 0x7f247c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2480006010 tx=0x7f2480000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.782 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.762+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f247809b330 con 0x7f2490102e00 2024-09-12T03:01:18.939 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:18.938+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2490062630 con 0x7f247c0a2bb0 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] boot 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi057 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.616 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 4273M 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:19.617 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:19.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T03:01:19.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] boot 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi057 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 4273M 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:19.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: osd.9 [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] boot 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: osdmap e65: 10 total, 10 up, 10 in 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi057 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 4273M 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:19.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:20.688 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:20 smithi181 ceph-mon[35320]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.689 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:20 smithi181 ceph-mon[35320]: osdmap e66: 10 total, 10 up, 10 in 2024-09-12T03:01:20.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:20 smithi204 ceph-mon[35013]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.819 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:20 smithi204 ceph-mon[35013]: osdmap e66: 10 total, 10 up, 10 in 2024-09-12T03:01:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:20 smithi057 ceph-mon[30361]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi181:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:20 smithi057 ceph-mon[30361]: osdmap e66: 10 total, 10 up, 10 in 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/1727955208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]': finished 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:01:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:21 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/1727955208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]': finished 2024-09-12T03:01:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:01:21.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:21 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/1727955208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]: dispatch 2024-09-12T03:01:21.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a48ee5-1571-4aa2-91ed-2f972739a137"}]': finished 2024-09-12T03:01:21.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T03:01:21.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:21 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:22 smithi057 ceph-mon[30361]: from='client.? 172.21.15.181:0/3337065785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:22.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:22 smithi204 ceph-mon[35013]: from='client.? 172.21.15.181:0/3337065785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:22 smithi181 ceph-mon[35320]: from='client.? 172.21.15.181:0/3337065785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:23 smithi057 ceph-mon[30361]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:23 smithi204 ceph-mon[35013]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:23 smithi181 ceph-mon[35320]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:25.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:25 smithi057 ceph-mon[30361]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:25 smithi204 ceph-mon[35013]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:25 smithi181 ceph-mon[35320]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:27 smithi181 ceph-mon[35320]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:27.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:27 smithi057 ceph-mon[30361]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:27 smithi204 ceph-mon[35013]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:29 smithi057 ceph-mon[30361]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:29 smithi204 ceph-mon[35013]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:29 smithi181 ceph-mon[35320]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:30.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:30 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:01:30.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:30 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:30.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:01:30.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:30 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:30.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:01:30.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:30 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:31.829 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:31 smithi181 ceph-mon[35320]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:31.829 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:31 smithi181 ceph-mon[35320]: Deploying daemon osd.10 on smithi181 2024-09-12T03:01:31.842 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:31 smithi204 ceph-mon[35013]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:31.842 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:31 smithi204 ceph-mon[35013]: Deploying daemon osd.10 on smithi181 2024-09-12T03:01:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:31 smithi057 ceph-mon[30361]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:31 smithi057 ceph-mon[30361]: Deploying daemon osd.10 on smithi181 2024-09-12T03:01:33.854 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:33 smithi181 ceph-mon[35320]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:33.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:33 smithi057 ceph-mon[30361]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:33 smithi204 ceph-mon[35013]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:35 smithi181 ceph-mon[35320]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:35.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:35 smithi057 ceph-mon[30361]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:35 smithi204 ceph-mon[35013]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:36.589 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:36.589 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:36.589 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:36.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:36.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:36.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:36.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:36.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:37.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:37 smithi057 ceph-mon[30361]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:37 smithi204 ceph-mon[35013]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:37 smithi181 ceph-mon[35320]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:39.679 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:39 smithi181 ceph-mon[35320]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:39.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:39 smithi057 ceph-mon[30361]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:39 smithi204 ceph-mon[35013]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:40.012 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.011+0000 7f248dffb640 1 -- 172.21.15.181:0/1844065476 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2490062630 con 0x7f247c0a2bb0 2024-09-12T03:01:40.012 INFO:teuthology.orchestra.run.smithi181.stdout:Created osd(s) 10 on host 'smithi181' 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 msgr2=0x7f247c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 0x7f247c0a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2480006010 tx=0x7f2480000f40 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 msgr2=0x7f2490198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f247800b4a0 tx=0x7f247800b960 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 shutdown_connections 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f247c0a2bb0 0x7f247c0a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24901037c0 0x7f249019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2490102e00 0x7f2490198490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 --2- 172.21.15.181:0/1844065476 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2490101c00 0x7f2490197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 >> 172.21.15.181:0/1844065476 conn(0x7f24900fd390 msgr2=0x7f2490106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 shutdown_connections 2024-09-12T03:01:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-12T03:01:40.013+0000 7f2496de9640 1 -- 172.21.15.181:0/1844065476 wait complete. 2024-09-12T03:01:40.842 DEBUG:teuthology.orchestra.run.smithi181:osd.10> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T03:01:40.845 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-09-12T03:01:40.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:01:41.106 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:01:41.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:41 smithi204 ceph-mon[35013]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:41.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:41 smithi181 ceph-mon[35320]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:41 smithi057 ceph-mon[30361]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:42.568 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:01:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:01:42.308+0000 7f529a973640 -1 osd.10 0 log_to_monitors true 2024-09-12T03:01:43.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:43 smithi181 ceph-mon[35320]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:43.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:43 smithi181 ceph-mon[35320]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:43 smithi181 ceph-mon[35320]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:43 smithi057 ceph-mon[30361]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:43.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:43 smithi057 ceph-mon[30361]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:43 smithi057 ceph-mon[30361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:43 smithi204 ceph-mon[35013]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:43.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:43 smithi204 ceph-mon[35013]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:43 smithi204 ceph-mon[35013]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:01:43.955 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-12T03:01:43.981 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-09-12T03:01:44.127 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-mon[35320]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:01:44.127 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-mon[35320]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:01:44.127 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-mon[35320]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.127 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:44.127 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-mon[35320]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.233 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:01:44.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:44 smithi204 ceph-mon[35013]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:01:44.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:44 smithi204 ceph-mon[35013]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:01:44.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:44 smithi204 ceph-mon[35013]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:44.356 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:44 smithi204 ceph-mon[35013]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:44 smithi057 ceph-mon[30361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:01:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:44 smithi057 ceph-mon[30361]: osdmap e68: 11 total, 10 up, 11 in 2024-09-12T03:01:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:44 smithi057 ceph-mon[30361]: from='osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:44 smithi057 ceph-mon[30361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]: dispatch 2024-09-12T03:01:44.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1848341280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641017a0 msgr2=0x7ff764109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1848341280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641017a0 0x7ff764109040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009ca0 tx=0x7ff75c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.738 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1848341280 shutdown_connections 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1848341280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641017a0 0x7ff764109040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1848341280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff764100de0 0x7ff764101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1848341280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7641020a0 0x7ff7641024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1848341280 >> 172.21.15.204:0/1848341280 conn(0x7ff7640fba40 msgr2=0x7ff7640fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1848341280 shutdown_connections 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1848341280 wait complete. 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 Processor -- start 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 -- start start 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.721+0000 7ff76b1ba640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 0x7ff7641983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff76410d890 con 0x7ff7641017a0 2024-09-12T03:01:44.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff76410d710 con 0x7ff764100de0 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff76410da10 con 0x7ff7641020a0 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff769757640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.204:58746/0 (socket says 172.21.15.204:58746) 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 -- 172.21.15.204:0/1862186572 learned_addr learned my addr 172.21.15.204:0/1862186572 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff769757640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55294/0 (socket says 172.21.15.204:55294) 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff763fff640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 0x7ff7641983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 -- 172.21.15.204:0/1862186572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 msgr2=0x7ff76419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 -- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 msgr2=0x7ff7641983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:44.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 0x7ff7641983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 -- 172.21.15.204:0/1862186572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76419fae0 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff763fff640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 0x7ff7641983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff769757640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff768f56640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c970 tx=0x7ff75800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758007c10 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76419fdd0 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff758007db0 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.722+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75800f8a0 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.723+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641a0230 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.723+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff764107fa0 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.724+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff75800fa80 con 0x7ff764100de0 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.725+0000 7ff761ffb640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 0x7ff7540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:44.741 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.725+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6414+0+0 (secure 0 0 0) 0x7ff7580cecb0 con 0x7ff764100de0 2024-09-12T03:01:44.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.725+0000 7ff763fff640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 0x7ff7540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:44.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.726+0000 7ff763fff640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 0x7ff7540a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7500058c0 tx=0x7ff750005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:44.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.729+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff75809c370 con 0x7ff764100de0 2024-09-12T03:01:44.907 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:01:44.907+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff764062630 con 0x7ff7540a2bb0 2024-09-12T03:01:44.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:01:44.699+0000 7f52968f6640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T03:01:44.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:01:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:01:44.711+0000 7f528e716640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:01:45.316 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:45.316 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:01:45.316 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:01:45.316 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.316 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi181 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-12T03:01:45.317 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='osd.10 ' entity='osd.10' 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:45.318 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:45 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi181 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='osd.10 ' entity='osd.10' 2024-09-12T03:01:45.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:45.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:45.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:45 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:01:45.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi181", "root=default"]}]': finished 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: osdmap e69: 11 total, 10 up, 11 in 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi181 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:01:45.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 4272M 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='osd.10 ' entity='osd.10' 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:01:45.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:45 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] boot 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:01:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:46 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:46.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T03:01:46.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T03:01:46.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:46.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] boot 2024-09-12T03:01:46.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:01:46.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:46 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:46.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T03:01:46.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T03:01:46.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: from='client.24398 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:01:46.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: osd.10 [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] boot 2024-09-12T03:01:46.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T03:01:46.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:46 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:01:47.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:47 smithi057 ceph-mon[30361]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:47 smithi057 ceph-mon[30361]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:01:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:47 smithi204 ceph-mon[35013]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:47 smithi204 ceph-mon[35013]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:01:47.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:47 smithi181 ceph-mon[35320]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:47.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:47 smithi181 ceph-mon[35320]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T03:01:48.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/3271032265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]': finished 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.204:0/4076299592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:48.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T03:01:48.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/3271032265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]': finished 2024-09-12T03:01:48.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:01:48.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:48.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.204:0/4076299592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:48.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: osdmap e72: 11 total, 11 up, 11 in 2024-09-12T03:01:48.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/3271032265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]: dispatch 2024-09-12T03:01:48.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d16ce7-b559-4958-9d46-03684f5c25ca"}]': finished 2024-09-12T03:01:48.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:01:48.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:48.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.204:0/4076299592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:01:49.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:49 smithi057 ceph-mon[30361]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:49.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:49 smithi057 ceph-mon[30361]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:01:49.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:49 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:49.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:49 smithi204 ceph-mon[35013]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:49.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:49 smithi204 ceph-mon[35013]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:01:49.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:49 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:49 smithi181 ceph-mon[35320]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:01:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:49 smithi181 ceph-mon[35320]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T03:01:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:49 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:01:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:51 smithi204 ceph-mon[35013]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:01:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:51 smithi181 ceph-mon[35320]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:01:51.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:51 smithi057 ceph-mon[30361]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 679 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:01:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:53 smithi204 ceph-mon[35013]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T03:01:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:53 smithi181 ceph-mon[35320]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T03:01:53.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:53 smithi057 ceph-mon[30361]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T03:01:55.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:55 smithi204 ceph-mon[35013]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:01:55.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:55 smithi057 ceph-mon[30361]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:01:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:55 smithi181 ceph-mon[35320]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T03:01:56.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:01:56.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:56 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:01:56.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:56 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:01:56.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:56 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:57.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:57 smithi204 ceph-mon[35013]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T03:01:57.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:57 smithi204 ceph-mon[35013]: Deploying daemon osd.11 on smithi204 2024-09-12T03:01:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:57 smithi181 ceph-mon[35320]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T03:01:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:57 smithi181 ceph-mon[35320]: Deploying daemon osd.11 on smithi204 2024-09-12T03:01:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:57 smithi057 ceph-mon[30361]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-12T03:01:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:57 smithi057 ceph-mon[30361]: Deploying daemon osd.11 on smithi204 2024-09-12T03:01:59.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:01:59 smithi204 ceph-mon[35013]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:01:59.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:01:59 smithi181 ceph-mon[35320]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:01:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:01:59 smithi057 ceph-mon[30361]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:02:01.529 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:01 smithi204 ceph-mon[35013]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T03:02:01.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:01 smithi181 ceph-mon[35320]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T03:02:01.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:01 smithi057 ceph-mon[30361]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-12T03:02:03.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:03 smithi204 ceph-mon[35013]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:03.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:03 smithi181 ceph-mon[35320]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:03.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:03 smithi057 ceph-mon[30361]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:05.614 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:05 smithi204 ceph-mon[35013]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:05 smithi181 ceph-mon[35320]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:05.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:05 smithi057 ceph-mon[30361]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:06.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.340+0000 7ff761ffb640 1 -- 172.21.15.204:0/1862186572 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff764062630 con 0x7ff7540a2bb0 2024-09-12T03:02:06.341 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 msgr2=0x7ff7540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 0x7ff7540a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7500058c0 tx=0x7ff750005850 comp rx=0 tx=0).stop 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 msgr2=0x7ff764197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c970 tx=0x7ff75800ce30 comp rx=0 tx=0).stop 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 shutdown_connections 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff7540a2bb0 0x7ff7540a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7641020a0 0x7ff76419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff7641017a0 0x7ff7641983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 --2- 172.21.15.204:0/1862186572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff764100de0 0x7ff764197eb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 >> 172.21.15.204:0/1862186572 conn(0x7ff7640fba40 msgr2=0x7ff764105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.342+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 shutdown_connections 2024-09-12T03:02:06.344 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:06.343+0000 7ff76b1ba640 1 -- 172.21.15.204:0/1862186572 wait complete. 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:06 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:06.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:06 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:06.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:06.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:06 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:07.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:07 smithi204 ceph-mon[35013]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:07 smithi057 ceph-mon[30361]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:07 smithi181 ceph-mon[35320]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:08.681 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:02:08.683 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:02:08.683 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:02:08.925 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:02:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:02:08.771+0000 7f946acbf640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:02:09.065 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:09.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 -- 172.21.15.57:0/820808299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c077df0 msgr2=0x7f098c078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 --2- 172.21.15.57:0/820808299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c077df0 0x7f098c078250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0978009900 tx=0x7f097802a410 comp rx=0 tx=0).stop 2024-09-12T03:02:09.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 -- 172.21.15.57:0/820808299 shutdown_connections 2024-09-12T03:02:09.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 --2- 172.21.15.57:0/820808299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c078790 0x7f098c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 --2- 172.21.15.57:0/820808299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c077df0 0x7f098c078250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 --2- 172.21.15.57:0/820808299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 0x7f098c076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 -- 172.21.15.57:0/820808299 >> 172.21.15.57:0/820808299 conn(0x7f098c0ff5b0 msgr2=0x7f098c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 -- 172.21.15.57:0/820808299 shutdown_connections 2024-09-12T03:02:09.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.603+0000 7f0994600640 1 -- 172.21.15.57:0/820808299 wait complete. 2024-09-12T03:02:09.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.604+0000 7f0994600640 1 Processor -- start 2024-09-12T03:02:09.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.604+0000 7f0994600640 1 -- start start 2024-09-12T03:02:09.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 0x7f098c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c077df0 0x7f098c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f098c113e80 con 0x7f098c078790 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f098c113d00 con 0x7f098c077df0 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0994600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f098c114000 con 0x7f098c076ba0 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35930/0 (socket says 172.21.15.57:35930) 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 -- 172.21.15.57:0/682966252 learned_addr learned my addr 172.21.15.57:0/682966252 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0991200640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c077df0 0x7f098c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0991c00640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 0x7f098c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 -- 172.21.15.57:0/682966252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 msgr2=0x7f098c19e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 0x7f098c19e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 -- 172.21.15.57:0/682966252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c077df0 msgr2=0x7f098c19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c077df0 0x7f098c19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.605+0000 7f0992600640 1 -- 172.21.15.57:0/682966252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c1a6110 con 0x7f098c078790 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f0992600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f098400eb10 tx=0x7f098400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984004b20 con 0x7f098c078790 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0984004cc0 con 0x7f098c078790 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c1a6400 con 0x7f098c078790 2024-09-12T03:02:09.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984010760 con 0x7f098c078790 2024-09-12T03:02:09.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.606+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c1a6910 con 0x7f098c078790 2024-09-12T03:02:09.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.607+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c110780 con 0x7f098c078790 2024-09-12T03:02:09.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.609+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0984005000 con 0x7f098c078790 2024-09-12T03:02:09.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.612+0000 7f098a000640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 0x7f09740a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.612+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7f09840cf200 con 0x7f098c078790 2024-09-12T03:02:09.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.612+0000 7f0991c00640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 0x7f09740a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.612+0000 7f0991c00640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 0x7f09740a50f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f098000bbe0 tx=0x7f098000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.614+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09840541b0 con 0x7f098c078790 2024-09-12T03:02:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:09 smithi204 ceph-mon[35013]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:09 smithi204 ceph-mon[35013]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:09.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:09 smithi204 ceph-mon[35013]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:09.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.804+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f098c062df0 con 0x7f098c078790 2024-09-12T03:02:09.804 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:09 smithi057 ceph-mon[30361]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:09.805 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:09 smithi057 ceph-mon[30361]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:09.805 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:09 smithi057 ceph-mon[30361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:09.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.805+0000 7f098a000640 1 -- 172.21.15.57:0/682966252 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f098409c6a0 con 0x7f098c078790 2024-09-12T03:02:09.805 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:09.806 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 msgr2=0x7f09740a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 0x7f09740a50f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f098000bbe0 tx=0x7f098000b390 comp rx=0 tx=0).stop 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 msgr2=0x7f098c1a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f098400eb10 tx=0x7f098400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 shutdown_connections 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f09740a2c30 0x7f09740a50f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f098c078790 0x7f098c1a5aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f098c077df0 0x7f098c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 --2- 172.21.15.57:0/682966252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c076ba0 0x7f098c19e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.807+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 >> 172.21.15.57:0/682966252 conn(0x7f098c0ff5b0 msgr2=0x7f098c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.808+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 shutdown_connections 2024-09-12T03:02:09.808 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:09.808+0000 7f0994600640 1 -- 172.21.15.57:0/682966252 wait complete. 2024-09-12T03:02:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:09 smithi181 ceph-mon[35320]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:09 smithi181 ceph-mon[35320]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:09 smithi181 ceph-mon[35320]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:02:10.655 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726110105,"num_in_osds":12,"osd_in_since":1726110107,"num_remapped_pgs":0} 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/682966252' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:10.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.893 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:02:10.632+0000 7f9466c42640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:02:10.893 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:02:10.646+0000 7f9462a6a640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/682966252' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:10.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:10.895 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:02:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: osdmap e75: 12 total, 11 up, 12 in 2024-09-12T03:02:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/682966252' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:10.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:11.656 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:02:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: Detected new or changed devices on smithi204 2024-09-12T03:02:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-12T03:02:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:02:11.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:02:11.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:11.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:11 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:12.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: Detected new or changed devices on smithi204 2024-09-12T03:02:12.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-12T03:02:12.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:02:12.010 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:02:12.011 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.011 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:11 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: Detected new or changed devices on smithi204 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: osdmap e76: 12 total, 11 up, 12 in 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:11 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 -- 172.21.15.57:0/138536334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4076ba0 msgr2=0x7f67c4076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 --2- 172.21.15.57:0/138536334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4076ba0 0x7f67c4076fa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009900 tx=0x7f67b402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:12.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 -- 172.21.15.57:0/138536334 shutdown_connections 2024-09-12T03:02:12.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 --2- 172.21.15.57:0/138536334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 0x7f67c4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 --2- 172.21.15.57:0/138536334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4077df0 0x7f67c4078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 --2- 172.21.15.57:0/138536334 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4076ba0 0x7f67c4076fa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 -- 172.21.15.57:0/138536334 >> 172.21.15.57:0/138536334 conn(0x7f67c40ff5b0 msgr2=0x7f67c41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 -- 172.21.15.57:0/138536334 shutdown_connections 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.605+0000 7f67ca800640 1 -- 172.21.15.57:0/138536334 wait complete. 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.606+0000 7f67ca800640 1 Processor -- start 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.606+0000 7f67ca800640 1 -- start start 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4077df0 0x7f67c419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 0x7f67c41a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4113f10 con 0x7f67c4077df0 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4113d90 con 0x7f67c4076ba0 2024-09-12T03:02:12.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67ca800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4114090 con 0x7f67c4078790 2024-09-12T03:02:12.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67c2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4077df0 0x7f67c419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67c3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67c3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41732/0 (socket says 172.21.15.57:41732) 2024-09-12T03:02:12.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67c3400640 1 -- 172.21.15.57:0/1667688764 learned_addr learned my addr 172.21.15.57:0/1667688764 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:12.610 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.607+0000 7f67c3e00640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 0x7f67c41a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.608+0000 7f67c3400640 1 -- 172.21.15.57:0/1667688764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 msgr2=0x7f67c41a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.608+0000 7f67c3400640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 0x7f67c41a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.608+0000 7f67c3400640 1 -- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4077df0 msgr2=0x7f67c419ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.608+0000 7f67c3400640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4077df0 0x7f67c419ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.608+0000 7f67c3400640 1 -- 172.21.15.57:0/1667688764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a60f0 con 0x7f67c4076ba0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67c3400640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67b4031440 tx=0x7f67b4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b403f030 con 0x7f67c4076ba0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a6380 con 0x7f67c4076ba0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b4003ce0 con 0x7f67c4076ba0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4035070 con 0x7f67c4076ba0 2024-09-12T03:02:12.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.609+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a68b0 con 0x7f67c4076ba0 2024-09-12T03:02:12.612 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.611+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c4110780 con 0x7f67c4076ba0 2024-09-12T03:02:12.614 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.614+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f67b4042030 con 0x7f67c4076ba0 2024-09-12T03:02:12.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.615+0000 7f67bbe00640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 0x7f67a80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.616+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f67b40ef1a0 con 0x7f67c4076ba0 2024-09-12T03:02:12.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.616+0000 7f67c2a00640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 0x7f67a80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.616+0000 7f67c2a00640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 0x7f67a80a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67c419f9e0 tx=0x7f67b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.617 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.617+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67b40bc500 con 0x7f67c4076ba0 2024-09-12T03:02:12.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.811+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f67c4062df0 con 0x7f67c4076ba0 2024-09-12T03:02:12.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.812+0000 7f67bbe00640 1 -- 172.21.15.57:0/1667688764 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f67b40bbe00 con 0x7f67c4076ba0 2024-09-12T03:02:12.813 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 msgr2=0x7f67a80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 0x7f67a80a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f67c419f9e0 tx=0x7f67b000b390 comp rx=0 tx=0).stop 2024-09-12T03:02:12.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 msgr2=0x7f67c419e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67b4031440 tx=0x7f67b4032ce0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 shutdown_connections 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f67a80a2bb0 0x7f67a80a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c4078790 0x7f67c41a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f67c4077df0 0x7f67c419ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 --2- 172.21.15.57:0/1667688764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67c4076ba0 0x7f67c419e4c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.814+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 >> 172.21.15.57:0/1667688764 conn(0x7f67c40ff5b0 msgr2=0x7f67c4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.815+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 shutdown_connections 2024-09-12T03:02:12.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:12.815+0000 7f67ca800640 1 -- 172.21.15.57:0/1667688764 wait complete. 2024-09-12T03:02:12.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: purged_snaps scrub starts 2024-09-12T03:02:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: purged_snaps scrub ok 2024-09-12T03:02:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] boot 2024-09-12T03:02:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:02:12.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:12 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: purged_snaps scrub starts 2024-09-12T03:02:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: purged_snaps scrub ok 2024-09-12T03:02:12.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:12.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] boot 2024-09-12T03:02:12.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:02:12.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:12 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:13.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: purged_snaps scrub starts 2024-09-12T03:02:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: purged_snaps scrub ok 2024-09-12T03:02:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: osd.11 [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] boot 2024-09-12T03:02:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:02:13.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:12 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:02:13.719 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726110131,"num_in_osds":12,"osd_in_since":1726110107,"num_remapped_pgs":0} 2024-09-12T03:02:13.720 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:02:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:13 smithi181 ceph-mon[35320]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:13 smithi181 ceph-mon[35320]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:02:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:13 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1667688764' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:14.090 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:14.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:13 smithi057 ceph-mon[30361]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:14.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:13 smithi057 ceph-mon[30361]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:02:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:13 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1667688764' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:13 smithi204 ceph-mon[35013]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:02:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:13 smithi204 ceph-mon[35013]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:02:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:13 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1667688764' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.763+0000 7f2633e00640 1 -- 172.21.15.57:0/1777252534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634108f60 msgr2=0x7f2634110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.763+0000 7f2633e00640 1 --2- 172.21.15.57:0/1777252534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634108f60 0x7f2634110ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f262c009ca0 tx=0x7f262c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.763+0000 7f2633e00640 1 -- 172.21.15.57:0/1777252534 shutdown_connections 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.763+0000 7f2633e00640 1 --2- 172.21.15.57:0/1777252534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634108f60 0x7f2634110ae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.764+0000 7f2633e00640 1 --2- 172.21.15.57:0/1777252534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f2634108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.764+0000 7f2633e00640 1 --2- 172.21.15.57:0/1777252534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634103e30 0x7f2634104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.764+0000 7f2633e00640 1 -- 172.21.15.57:0/1777252534 >> 172.21.15.57:0/1777252534 conn(0x7f26340ff8d0 msgr2=0x7f2634101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.764+0000 7f2633e00640 1 -- 172.21.15.57:0/1777252534 shutdown_connections 2024-09-12T03:02:14.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.764+0000 7f2633e00640 1 -- 172.21.15.57:0/1777252534 wait complete. 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 Processor -- start 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 -- start start 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f263419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 0x7f263419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2634113fc0 con 0x7f2634103e30 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2634113e40 con 0x7f2634108f60 2024-09-12T03:02:14.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2633e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2634114140 con 0x7f26341047e0 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.765+0000 7f2632a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2632a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35972/0 (socket says 172.21.15.57:35972) 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2632a00640 1 -- 172.21.15.57:0/2140042049 learned_addr learned my addr 172.21.15.57:0/2140042049 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f262be00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f263419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 0x7f263419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 -- 172.21.15.57:0/2140042049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 msgr2=0x7f263419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f263419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 -- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 msgr2=0x7f263419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2633400640 1 -- 172.21.15.57:0/2140042049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263419f3d0 con 0x7f2634108f60 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f262be00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f263419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:14.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.766+0000 7f2632a00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f2633400640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 0x7f263419ecf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f262c02ade0 tx=0x7f262c0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c003b70 con 0x7f2634108f60 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263419f660 con 0x7f2634108f60 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c003d10 con 0x7f2634108f60 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c02fcb0 con 0x7f2634108f60 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.767+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26341a6fa0 con 0x7f2634108f60 2024-09-12T03:02:14.769 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.768+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634104210 con 0x7f2634108f60 2024-09-12T03:02:14.770 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.770+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f262c004050 con 0x7f2634108f60 2024-09-12T03:02:14.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.771+0000 7f262b400640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:14.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.771+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f262c0efd40 con 0x7f2634108f60 2024-09-12T03:02:14.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.772+0000 7f2632a00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:14.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.772+0000 7f2632a00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 0x7f26180a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2624011550 tx=0x7f2624005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:14.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.775+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f262c0bd0a0 con 0x7f2634108f60 2024-09-12T03:02:14.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.959+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f263410c1a0 con 0x7f2634108f60 2024-09-12T03:02:14.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.961+0000 7f262b400640 1 -- 172.21.15.57:0/2140042049 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18455 (secure 0 0 0) 0x7f262c0bc9a0 con 0x7f2634108f60 2024-09-12T03:02:14.962 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:14.962 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":78,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:02:12.642687+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 msgr2=0x7f26180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 0x7f26180a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2624011550 tx=0x7f2624005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 msgr2=0x7f263419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 0x7f263419ecf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f262c02ade0 tx=0x7f262c0319f0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 shutdown_connections 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f26180a2c00 0x7f26180a50c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2634108f60 0x7f263419ecf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26341047e0 0x7f263419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 --2- 172.21.15.57:0/2140042049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2634103e30 0x7f263419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 >> 172.21.15.57:0/2140042049 conn(0x7f26340ff8d0 msgr2=0x7f2634100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 shutdown_connections 2024-09-12T03:02:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:14.963+0000 7f2633e00640 1 -- 172.21.15.57:0/2140042049 wait complete. 2024-09-12T03:02:15.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:15 smithi057 ceph-mon[30361]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:15.850 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:15 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2140042049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:15.892 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:58:06.250728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:02:15.892 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:02:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:15 smithi181 ceph-mon[35320]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:15 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2140042049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:15 smithi204 ceph-mon[35013]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:15 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2140042049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:02:16.264 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:16.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 -- 172.21.15.57:0/1826517093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac103e30 msgr2=0x7f90ac104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 --2- 172.21.15.57:0/1826517093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac103e30 0x7f90ac104210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f909c009900 tx=0x7f909c02a410 comp rx=0 tx=0).stop 2024-09-12T03:02:16.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 -- 172.21.15.57:0/1826517093 shutdown_connections 2024-09-12T03:02:16.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 --2- 172.21.15.57:0/1826517093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac108f60 0x7f90ac110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 --2- 172.21.15.57:0/1826517093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 0x7f90ac108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 --2- 172.21.15.57:0/1826517093 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac103e30 0x7f90ac104210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 -- 172.21.15.57:0/1826517093 >> 172.21.15.57:0/1826517093 conn(0x7f90ac0ff910 msgr2=0x7f90ac101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:16.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 -- 172.21.15.57:0/1826517093 shutdown_connections 2024-09-12T03:02:16.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.847+0000 7f90ab200640 1 -- 172.21.15.57:0/1826517093 wait complete. 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.848+0000 7f90ab200640 1 Processor -- start 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.848+0000 7f90ab200640 1 -- start start 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac103e30 0x7f90ac19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 0x7f90ac19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac113fc0 con 0x7f90ac108f60 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac113e40 con 0x7f90ac103e30 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90ab200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac114140 con 0x7f90ac1047e0 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35992/0 (socket says 172.21.15.57:35992) 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 -- 172.21.15.57:0/2613317177 learned_addr learned my addr 172.21.15.57:0/2613317177 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90a9400640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 0x7f90ac19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90a9e00640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac103e30 0x7f90ac19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 -- 172.21.15.57:0/2613317177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 msgr2=0x7f90ac19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 0x7f90ac19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 -- 172.21.15.57:0/2613317177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac103e30 msgr2=0x7f90ac19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac103e30 0x7f90ac19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:16.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.849+0000 7f90aa800640 1 -- 172.21.15.57:0/2613317177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac19f440 con 0x7f90ac108f60 2024-09-12T03:02:16.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90aa800640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f90a400eaa0 tx=0x7f90a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:16.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400cdb0 con 0x7f90ac108f60 2024-09-12T03:02:16.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4005020 con 0x7f90ac108f60 2024-09-12T03:02:16.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40116b0 con 0x7f90ac108f60 2024-09-12T03:02:16.852 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac19f730 con 0x7f90ac108f60 2024-09-12T03:02:16.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.850+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a7010 con 0x7f90ac108f60 2024-09-12T03:02:16.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.852+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac105310 con 0x7f90ac108f60 2024-09-12T03:02:16.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.853+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f90a4005560 con 0x7f90ac108f60 2024-09-12T03:02:16.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.855+0000 7f90a2a00640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 0x7f90900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:16.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.855+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f90a40ced10 con 0x7f90ac108f60 2024-09-12T03:02:16.855 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.855+0000 7f90a9e00640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 0x7f90900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:16.856 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.856+0000 7f90a9e00640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 0x7f90900a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f909c009230 tx=0x7f909c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:16.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:16.859+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90a40d4020 con 0x7f90ac108f60 2024-09-12T03:02:17.041 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.041+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90ac02d7c0 con 0x7f90ac108f60 2024-09-12T03:02:17.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.041+0000 7f90a2a00640 1 -- 172.21.15.57:0/2613317177 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f90a4016030 con 0x7f90ac108f60 2024-09-12T03:02:17.042 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-09-12T03:02:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 msgr2=0x7f90900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 0x7f90900a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f909c009230 tx=0x7f909c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:02:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 msgr2=0x7f90ac19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f90a400eaa0 tx=0x7f90a400ef60 comp rx=0 tx=0).stop 2024-09-12T03:02:17.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 shutdown_connections 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90900a2c00 0x7f90900a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90ac108f60 0x7f90ac19ed60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac1047e0 0x7f90ac19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 --2- 172.21.15.57:0/2613317177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90ac103e30 0x7f90ac19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.044+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 >> 172.21.15.57:0/2613317177 conn(0x7f90ac0ff910 msgr2=0x7f90ac100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.045+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 shutdown_connections 2024-09-12T03:02:17.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:17.045+0000 7f90ab200640 1 -- 172.21.15.57:0/2613317177 wait complete. 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.a on smithi057 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.d on smithi057 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.g on smithi057 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.j on smithi057 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.b on smithi181 2024-09-12T03:02:17.775 INFO:tasks.cephadm:Adding mds.e on smithi181 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.h on smithi181 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.k on smithi181 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-09-12T03:02:17.776 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-09-12T03:02:17.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-09-12T03:02:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:17 smithi181 ceph-mon[35320]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:17 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2613317177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:18.037 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:02:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:17 smithi057 ceph-mon[30361]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:17 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2613317177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:17 smithi204 ceph-mon[35013]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:17 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2613317177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:02:18.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 -- 172.21.15.204:0/2849246691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5354104ff0 msgr2=0x7f53541073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 --2- 172.21.15.204:0/2849246691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5354104ff0 0x7f53541073e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5350009900 tx=0x7f535002a410 comp rx=0 tx=0).stop 2024-09-12T03:02:18.568 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 -- 172.21.15.204:0/2849246691 shutdown_connections 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 --2- 172.21.15.204:0/2849246691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354107920 0x7f5354109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 --2- 172.21.15.204:0/2849246691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5354104ff0 0x7f53541073e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 --2- 172.21.15.204:0/2849246691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f53541026c0 0x7f5354104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 -- 172.21.15.204:0/2849246691 >> 172.21.15.204:0/2849246691 conn(0x7f53540fc280 msgr2=0x7f53540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 -- 172.21.15.204:0/2849246691 shutdown_connections 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.553+0000 7f535b8b2640 1 -- 172.21.15.204:0/2849246691 wait complete. 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 Processor -- start 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 -- start start 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354104ff0 0x7f5354198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 0x7f535419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f535410d8a0 con 0x7f5354107920 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f535410d720 con 0x7f5354104ff0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535b8b2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f535410da20 con 0x7f53541026c0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535964e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535964e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55756/0 (socket says 172.21.15.204:55756) 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f535964e640 1 -- 172.21.15.204:0/1501888471 learned_addr learned my addr 172.21.15.204:0/1501888471 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.554+0000 7f5358e4d640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354104ff0 0x7f5354198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f5359e4f640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 0x7f535419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 -- 172.21.15.204:0/1501888471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354104ff0 msgr2=0x7f5354198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354104ff0 0x7f5354198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 -- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 msgr2=0x7f535419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 0x7f535419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 -- 172.21.15.204:0/1501888471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535419fb50 con 0x7f53541026c0 2024-09-12T03:02:18.570 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f5359e4f640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 0x7f535419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535964e640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535419fe40 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5344009c30 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400f6b0 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.555+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53541a02a0 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.556+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53540fdd70 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.557+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f53440109d0 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.558+0000 7f534a7fc640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 0x7f53400a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.558+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f53440ce7c0 con 0x7f53541026c0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.558+0000 7f5358e4d640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 0x7f53400a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.559+0000 7f5358e4d640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 0x7f53400a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f535002aa00 tx=0x7f53500315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.571 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.562+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f534409bbc0 con 0x7f53541026c0 2024-09-12T03:02:18.742 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.741+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f53540627f0 con 0x7f53400a2bb0 2024-09-12T03:02:18.750 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.750+0000 7f534a7fc640 1 -- 172.21.15.204:0/1501888471 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f53540627f0 con 0x7f53400a2bb0 2024-09-12T03:02:18.750 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-09-12T03:02:18.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 msgr2=0x7f53400a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.752 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 0x7f53400a5070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f535002aa00 tx=0x7f53500315b0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 msgr2=0x7f5354197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f534400d950 tx=0x7f534400de10 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 shutdown_connections 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53400a2bb0 0x7f53400a5070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5354107920 0x7f535419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5354104ff0 0x7f5354198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 --2- 172.21.15.204:0/1501888471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53541026c0 0x7f5354197f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 >> 172.21.15.204:0/1501888471 conn(0x7f53540fc280 msgr2=0x7f5354102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 shutdown_connections 2024-09-12T03:02:18.753 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-12T03:02:18.752+0000 7f535b8b2640 1 -- 172.21.15.204:0/1501888471 wait complete. 2024-09-12T03:02:19.494 DEBUG:teuthology.orchestra.run.smithi057:mds.a> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:02:19.499 DEBUG:teuthology.orchestra.run.smithi057:mds.d> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:02:19.544 DEBUG:teuthology.orchestra.run.smithi057:mds.g> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:02:19.589 DEBUG:teuthology.orchestra.run.smithi057:mds.j> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:02:19.634 DEBUG:teuthology.orchestra.run.smithi181:mds.b> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:02:19.636 DEBUG:teuthology.orchestra.run.smithi181:mds.e> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:02:19.679 DEBUG:teuthology.orchestra.run.smithi181:mds.h> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:19.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:19 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:19.722 DEBUG:teuthology.orchestra.run.smithi181:mds.k> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:02:19.765 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:02:19.767 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:02:19.810 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:02:19.853 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:19 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:19.896 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:02:19.896 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:19.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:19.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:19.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:19 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:20.265 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:20.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.774+0000 7fdc5ee00640 1 -- 172.21.15.57:0/2888643208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 msgr2=0x7fdc581124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.774+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/2888643208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581124a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009900 tx=0x7fdc4402a480 comp rx=0 tx=0).stop 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 -- 172.21.15.57:0/2888643208 shutdown_connections 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/2888643208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc581129e0 0x7fdc58114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/2888643208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581124a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/2888643208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc58074590 0x7fdc58074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 -- 172.21.15.57:0/2888643208 >> 172.21.15.57:0/2888643208 conn(0x7fdc580fd710 msgr2=0x7fdc580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 -- 172.21.15.57:0/2888643208 shutdown_connections 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.775+0000 7fdc5ee00640 1 -- 172.21.15.57:0/2888643208 wait complete. 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 Processor -- start 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 -- start start 2024-09-12T03:02:20.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc581129e0 0x7fdc5819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58118400 con 0x7fdc58074eb0 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58118280 con 0x7fdc581129e0 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58118580 con 0x7fdc58074590 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5d000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:46120/0 (socket says 172.21.15.57:46120) 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.776+0000 7fdc5da00640 1 -- 172.21.15.57:0/4004870388 learned_addr learned my addr 172.21.15.57:0/4004870388 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5e400640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc581129e0 0x7fdc5819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 -- 172.21.15.57:0/4004870388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc581129e0 msgr2=0x7fdc5819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc581129e0 0x7fdc5819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 -- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 msgr2=0x7fdc581a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 -- 172.21.15.57:0/4004870388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5819ef30 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc5da00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ebc0 tx=0x7fdc4c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.777+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c010630 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.778+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5819f220 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.778+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4c010cb0 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.778+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c018bd0 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.778+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc581ab470 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.779+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc58075cd0 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.780+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc4c0107d0 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.782+0000 7fdc56000640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 0x7fdc400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.782+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fdc4c0cf260 con 0x7fdc58074590 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.782+0000 7fdc5d000640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 0x7fdc400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.783+0000 7fdc5d000640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 0x7fdc400a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc44005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:20.786+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc4c09bf70 con 0x7fdc58074590 2024-09-12T03:02:20.825 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:20 smithi204 ceph-mon[35013]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:20.825 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:20 smithi204 ceph-mon[35013]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-12T03:02:20.825 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:20 smithi204 ceph-mon[35013]: Deploying daemon mds.l on smithi204 2024-09-12T03:02:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:20 smithi181 ceph-mon[35320]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:20 smithi181 ceph-mon[35320]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-12T03:02:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:20 smithi181 ceph-mon[35320]: Deploying daemon mds.l on smithi204 2024-09-12T03:02:21.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.008+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc58104e20 con 0x7fdc58074590 2024-09-12T03:02:21.009 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:20 smithi057 ceph-mon[30361]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:02:21.009 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:20 smithi057 ceph-mon[30361]: Saving service mds.all spec with placement smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-12T03:02:21.009 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:20 smithi057 ceph-mon[30361]: Deploying daemon mds.l on smithi204 2024-09-12T03:02:21.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.008+0000 7fdc56000640 1 -- 172.21.15.57:0/4004870388 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fdc4c0a0270 con 0x7fdc58074590 2024-09-12T03:02:21.010 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:21.010 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":1,"btime":"2024-09-12T02:54:45:449413+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:02:21.010 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 1 2024-09-12T03:02:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.010+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 msgr2=0x7fdc400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.010+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 0x7fdc400a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc44000c00 tx=0x7fdc44005b50 comp rx=0 tx=0).stop 2024-09-12T03:02:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.010+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 msgr2=0x7fdc581a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:21.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.010+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00ebc0 tx=0x7fdc4c00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 shutdown_connections 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdc400a2b80 0x7fdc400a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc581129e0 0x7fdc5819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc58074eb0 0x7fdc581a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 --2- 172.21.15.57:0/4004870388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc58074590 0x7fdc581a4610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 >> 172.21.15.57:0/4004870388 conn(0x7fdc580fd710 msgr2=0x7fdc580feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 shutdown_connections 2024-09-12T03:02:21.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:21.011+0000 7fdc5ee00640 1 -- 172.21.15.57:0/4004870388 wait complete. 2024-09-12T03:02:21.394 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 systemd[1]: Starting Ceph mds.l for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:21.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:21 smithi204 ceph-mon[35013]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:21.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:21 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/4004870388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:21.802 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 podman[64814]: 2024-09-12 03:02:21.409921408 +0000 UTC m=+0.015882341 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:21.802 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 podman[64814]: 2024-09-12 03:02:21.536300176 +0000 UTC m=+0.142261141 container create a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T03:02:21.802 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 podman[64814]: 2024-09-12 03:02:21.802679283 +0000 UTC m=+0.408640206 container init a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T03:02:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:21 smithi181 ceph-mon[35320]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:21 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/4004870388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:22.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:21 smithi057 ceph-mon[30361]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:21 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4004870388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:22.143 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 podman[64814]: 2024-09-12 03:02:21.805785229 +0000 UTC m=+0.411746159 container start a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:02:22.144 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 ceph-mds[64833]: -- 172.21.15.204:0/2396612796 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55759383f860 con 0x55759381d400 2024-09-12T03:02:22.144 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l[64829]: starting mds.l at 2024-09-12T03:02:22.144 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 bash[64814]: a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f 2024-09-12T03:02:22.144 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:02:21 smithi204 systemd[1]: Started Ceph mds.l for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6832/3741354654,v1:172.21.15.204:6833/3741354654] up:boot 2024-09-12T03:02:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: fsmap 1 up:standby 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:22 smithi181 ceph-mon[35320]: Deploying daemon mds.k on smithi181 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6832/3741354654,v1:172.21.15.204:6833/3741354654] up:boot 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: fsmap 1 up:standby 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:22 smithi057 ceph-mon[30361]: Deploying daemon mds.k on smithi181 2024-09-12T03:02:23.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:23.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6832/3741354654,v1:172.21.15.204:6833/3741354654] up:boot 2024-09-12T03:02:23.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: fsmap 1 up:standby 2024-09-12T03:02:23.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:23.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:22 smithi204 ceph-mon[35013]: Deploying daemon mds.k on smithi181 2024-09-12T03:02:23.758 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:24.141 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:24.203 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 systemd[1]: Starting Ceph mds.k for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:24.522 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 podman[64476]: 2024-09-12 03:02:24.155006613 +0000 UTC m=+0.015722792 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:24.522 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 podman[64476]: 2024-09-12 03:02:24.270068055 +0000 UTC m=+0.130784222 container create b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:02:24.523 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 podman[64476]: 2024-09-12 03:02:24.522911003 +0000 UTC m=+0.383627171 container init b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 -- 172.21.15.57:0/2003755254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05641019d0 msgr2=0x7f0564101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 --2- 172.21.15.57:0/2003755254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05641019d0 0x7f0564101e30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0550009930 tx=0x7f055002a420 comp rx=0 tx=0).stop 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 -- 172.21.15.57:0/2003755254 shutdown_connections 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 --2- 172.21.15.57:0/2003755254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0564102370 0x7f056410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 --2- 172.21.15.57:0/2003755254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05641019d0 0x7f0564101e30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 --2- 172.21.15.57:0/2003755254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641079d0 0x7f0564107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 -- 172.21.15.57:0/2003755254 >> 172.21.15.57:0/2003755254 conn(0x7f05640fd710 msgr2=0x7f05640ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 -- 172.21.15.57:0/2003755254 shutdown_connections 2024-09-12T03:02:24.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.686+0000 7f0563200640 1 -- 172.21.15.57:0/2003755254 wait complete. 2024-09-12T03:02:24.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.687+0000 7f0563200640 1 Processor -- start 2024-09-12T03:02:24.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.687+0000 7f0563200640 1 -- start start 2024-09-12T03:02:24.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0564102370 0x7f0564198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05641079d0 0x7f056419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111e20 con 0x7f0564102370 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111ca0 con 0x7f05641019d0 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0563200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111fa0 con 0x7f05641079d0 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0561e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0561400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0564102370 0x7f0564198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0561e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:39358/0 (socket says 172.21.15.57:39358) 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0561e00640 1 -- 172.21.15.57:0/3742220944 learned_addr learned my addr 172.21.15.57:0/3742220944 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0562800640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05641079d0 0x7f056419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.688+0000 7f0561e00640 1 -- 172.21.15.57:0/3742220944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05641079d0 msgr2=0x7f056419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0561e00640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05641079d0 0x7f056419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0561e00640 1 -- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0564102370 msgr2=0x7f0564198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0561e00640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0564102370 0x7f0564198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0561e00640 1 -- 172.21.15.57:0/3742220944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056419d2b0 con 0x7f05641019d0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0561e00640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f055400d950 tx=0x7f055400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554017070 con 0x7f05641019d0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0554009c30 con 0x7f05641019d0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400f6b0 con 0x7f05641019d0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056419d5a0 con 0x7f05641019d0 2024-09-12T03:02:24.692 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.689+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0564074420 con 0x7f05641019d0 2024-09-12T03:02:24.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.691+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0564103110 con 0x7f05641019d0 2024-09-12T03:02:24.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.692+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05540109d0 con 0x7f05641019d0 2024-09-12T03:02:24.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.694+0000 7f055aa00640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 0x7f05480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.694+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f05540ce710 con 0x7f05641019d0 2024-09-12T03:02:24.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.694+0000 7f0561400640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 0x7f05480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.695+0000 7f0561400640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 0x7f05480a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0550006010 tx=0x7f055003c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.698+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0554009de0 con 0x7f05641019d0 2024-09-12T03:02:24.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.923+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0564101e30 con 0x7f05641019d0 2024-09-12T03:02:24.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.924+0000 7f055aa00640 1 -- 172.21.15.57:0/3742220944 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f055409b420 con 0x7f05641019d0 2024-09-12T03:02:24.925 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:24.925 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":3,"btime":"2024-09-12T03:02:24:842182+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:02:24.925 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 3 2024-09-12T03:02:24.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 msgr2=0x7f05480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 0x7f05480a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0550006010 tx=0x7f055003c040 comp rx=0 tx=0).stop 2024-09-12T03:02:24.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 msgr2=0x7f0564198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f055400d950 tx=0x7f055400de10 comp rx=0 tx=0).stop 2024-09-12T03:02:24.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 shutdown_connections 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05480a2c00 0x7f05480a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05641079d0 0x7f056419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0564102370 0x7f0564198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 --2- 172.21.15.57:0/3742220944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05641019d0 0x7f0564198300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.926+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 >> 172.21.15.57:0/3742220944 conn(0x7f05640fd710 msgr2=0x7f05640fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.927+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 shutdown_connections 2024-09-12T03:02:24.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:24.927+0000 7f0563200640 1 -- 172.21.15.57:0/3742220944 wait complete. 2024-09-12T03:02:24.953 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 podman[64476]: 2024-09-12 03:02:24.526032428 +0000 UTC m=+0.386748608 container start b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:02:24.953 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 ceph-mds[64495]: -- 172.21.15.181:0/461827277 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8602a9860 con 0x55c860287000 2024-09-12T03:02:24.953 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k[64491]: starting mds.k at 2024-09-12T03:02:24.953 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 bash[64476]: b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c 2024-09-12T03:02:24.953 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:02:24 smithi181 systemd[1]: Started Ceph mds.k for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6832/1889756926,v1:172.21.15.181:6833/1889756926] up:boot 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: fsmap 2 up:standby 2024-09-12T03:02:25.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:02:25.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:25 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3742220944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:25.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:25.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6832/1889756926,v1:172.21.15.181:6833/1889756926] up:boot 2024-09-12T03:02:25.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: fsmap 2 up:standby 2024-09-12T03:02:25.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:02:25.955 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:25 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3742220944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6832/1889756926,v1:172.21.15.181:6833/1889756926] up:boot 2024-09-12T03:02:26.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: fsmap 2 up:standby 2024-09-12T03:02:26.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:02:26.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:25 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3742220944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:26.947 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:26 smithi181 ceph-mon[35320]: Deploying daemon mds.j on smithi057 2024-09-12T03:02:27.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:26 smithi057 ceph-mon[30361]: Deploying daemon mds.j on smithi057 2024-09-12T03:02:27.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:26 smithi204 ceph-mon[35013]: Deploying daemon mds.j on smithi057 2024-09-12T03:02:27.645 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:27.831 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:27 smithi057 ceph-mon[30361]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:27 smithi181 ceph-mon[35320]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:28.105 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:27 smithi057 podman[69561]: 2024-09-12 03:02:27.859796441 +0000 UTC m=+0.152558534 container create c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:02:28.105 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 podman[69561]: 2024-09-12 03:02:28.105186061 +0000 UTC m=+0.397948155 container init c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:02:28.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:27 smithi204 ceph-mon[35013]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:28.340 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:28.369 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 podman[69561]: 2024-09-12 03:02:28.111644218 +0000 UTC m=+0.404406313 container start c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:02:28.370 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 ceph-mds[69604]: -- 172.21.15.57:0/260146057 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d787d1860 con 0x564d787af000 2024-09-12T03:02:28.370 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j[69589]: starting mds.j at 2024-09-12T03:02:28.370 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 bash[69561]: c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f 2024-09-12T03:02:28.370 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:02:28 smithi057 systemd[1]: Started Ceph mds.j for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 -- 172.21.15.57:0/1592227180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 msgr2=0x7fa4241043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 --2- 172.21.15.57:0/1592227180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa4241043d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa418009900 tx=0x7fa41802a410 comp rx=0 tx=0).stop 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 -- 172.21.15.57:0/1592227180 shutdown_connections 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 --2- 172.21.15.57:0/1592227180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 0x7fa424110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 --2- 172.21.15.57:0/1592227180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 0x7fa4241089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 --2- 172.21.15.57:0/1592227180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa4241043d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 -- 172.21.15.57:0/1592227180 >> 172.21.15.57:0/1592227180 conn(0x7fa4240ffa90 msgr2=0x7fa424101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 -- 172.21.15.57:0/1592227180 shutdown_connections 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.949+0000 7fa42c400640 1 -- 172.21.15.57:0/1592227180 wait complete. 2024-09-12T03:02:28.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.950+0000 7fa42c400640 1 Processor -- start 2024-09-12T03:02:28.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.950+0000 7fa42c400640 1 -- start start 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 0x7fa42419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 0x7fa42419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa424114170 con 0x7fa424103ff0 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa424113ff0 con 0x7fa424109120 2024-09-12T03:02:28.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241142f0 con 0x7fa4241049a0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa429a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa429a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54998/0 (socket says 172.21.15.57:54998) 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa429a00640 1 -- 172.21.15.57:0/85750370 learned_addr learned my addr 172.21.15.57:0/85750370 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa429000640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 0x7fa42419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.951+0000 7fa42a400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 0x7fa42419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 -- 172.21.15.57:0/85750370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 msgr2=0x7fa42419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 0x7fa42419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 -- 172.21.15.57:0/85750370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 msgr2=0x7fa42419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 0x7fa42419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 -- 172.21.15.57:0/85750370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419f620 con 0x7fa424103ff0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa429a00640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa418004770 tx=0x7fa418003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41803f070 con 0x7fa424103ff0 2024-09-12T03:02:28.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa418031ca0 con 0x7fa424103ff0 2024-09-12T03:02:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419f8b0 con 0x7fa424103ff0 2024-09-12T03:02:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.952+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418035030 con 0x7fa424103ff0 2024-09-12T03:02:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.953+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241a71f0 con 0x7fa424103ff0 2024-09-12T03:02:28.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.954+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4241043d0 con 0x7fa424103ff0 2024-09-12T03:02:28.956 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.955+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa418042070 con 0x7fa424103ff0 2024-09-12T03:02:28.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.957+0000 7fa422000640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 0x7fa40c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:28.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.957+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fa4180ef2e0 con 0x7fa424103ff0 2024-09-12T03:02:28.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.957+0000 7fa429000640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 0x7fa40c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:28.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.958+0000 7fa429000640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 0x7fa40c0a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4240fff40 tx=0x7fa41000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:28.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:28.961+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4180bbff0 con 0x7fa424103ff0 2024-09-12T03:02:29.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.189+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa42410c300 con 0x7fa424103ff0 2024-09-12T03:02:29.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.189+0000 7fa422000640 1 -- 172.21.15.57:0/85750370 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fa4180c02f0 con 0x7fa424103ff0 2024-09-12T03:02:29.191 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:29.191 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":4,"btime":"2024-09-12T03:02:28:685034+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-09-12T03:02:29.191 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 4 2024-09-12T03:02:29.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 msgr2=0x7fa40c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 0x7fa40c0a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4240fff40 tx=0x7fa41000b410 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 msgr2=0x7fa42419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa418004770 tx=0x7fa418003c10 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 shutdown_connections 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa40c0a2c00 0x7fa40c0a50c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa424109120 0x7fa42419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4241049a0 0x7fa42419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 --2- 172.21.15.57:0/85750370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa424103ff0 0x7fa42419a670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 >> 172.21.15.57:0/85750370 conn(0x7fa4240ffa90 msgr2=0x7fa424100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 shutdown_connections 2024-09-12T03:02:29.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:29.192+0000 7fa42c400640 1 -- 172.21.15.57:0/85750370 wait complete. 2024-09-12T03:02:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: Deploying daemon mds.d on smithi057 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6834/4044082469,v1:172.21.15.57:6835/4044082469] up:boot 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: fsmap 3 up:standby 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:02:29.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:29 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/85750370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:29.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:29.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: Deploying daemon mds.d on smithi057 2024-09-12T03:02:29.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6834/4044082469,v1:172.21.15.57:6835/4044082469] up:boot 2024-09-12T03:02:29.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: fsmap 3 up:standby 2024-09-12T03:02:29.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:02:29.646 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:29 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/85750370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: Deploying daemon mds.d on smithi057 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6834/4044082469,v1:172.21.15.57:6835/4044082469] up:boot 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: fsmap 3 up:standby 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:02:29.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:29 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/85750370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:31.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:31 smithi057 ceph-mon[30361]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:31.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:31 smithi204 ceph-mon[35013]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:31 smithi181 ceph-mon[35320]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:31.971 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:31 smithi057 systemd[1]: Starting Ceph mds.d for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:32.237 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:31 smithi057 podman[69947]: 2024-09-12 03:02:31.874709561 +0000 UTC m=+0.027216345 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:32.237 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:31 smithi057 podman[69947]: 2024-09-12 03:02:31.981681077 +0000 UTC m=+0.134187851 container create f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:02:32.237 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 podman[69947]: 2024-09-12 03:02:32.19275691 +0000 UTC m=+0.345263707 container init f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:02:32.237 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 podman[69947]: 2024-09-12 03:02:32.19912769 +0000 UTC m=+0.351634470 container start f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:02:32.389 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:32.619 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 ceph-mds[69966]: -- 172.21.15.57:0/763671032 <== mon.0 v2:172.21.15.57:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe70e9f860 con 0x55fe70e7d000 2024-09-12T03:02:32.620 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d[69962]: starting mds.d at 2024-09-12T03:02:32.620 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 bash[69947]: f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 2024-09-12T03:02:32.620 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:02:32 smithi057 systemd[1]: Started Ceph mds.d for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:32.767 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:33.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 -- 172.21.15.57:0/4065172672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b8102360 msgr2=0x7f68b810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 --2- 172.21.15.57:0/4065172672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b8102360 0x7f68b810e890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009ca0 tx=0x7f68b002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 -- 172.21.15.57:0/4065172672 shutdown_connections 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 --2- 172.21.15.57:0/4065172672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b8102360 0x7f68b810e890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 --2- 172.21.15.57:0/4065172672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b81019c0 0x7f68b8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 --2- 172.21.15.57:0/4065172672 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81079c0 0x7f68b8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 -- 172.21.15.57:0/4065172672 >> 172.21.15.57:0/4065172672 conn(0x7f68b80fd740 msgr2=0x7f68b80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 -- 172.21.15.57:0/4065172672 shutdown_connections 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.310+0000 7f68be000640 1 -- 172.21.15.57:0/4065172672 wait complete. 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.311+0000 7f68be000640 1 Processor -- start 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.311+0000 7f68be000640 1 -- start start 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8102360 0x7f68b819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 0x7f68b819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:33.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8111e10 con 0x7f68b81019c0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8111c90 con 0x7f68b81079c0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68be000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b8111f90 con 0x7f68b8102360 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55036/0 (socket says 172.21.15.57:55036) 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 -- 172.21.15.57:0/1106639073 learned_addr learned my addr 172.21.15.57:0/1106639073 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bd600640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 0x7f68b819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68b7e00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8102360 0x7f68b819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 -- 172.21.15.57:0/1106639073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8102360 msgr2=0x7f68b819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8102360 0x7f68b819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 -- 172.21.15.57:0/1106639073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 msgr2=0x7f68b819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 0x7f68b819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bcc00640 1 -- 172.21.15.57:0/1106639073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819f460 con 0x7f68b81019c0 2024-09-12T03:02:33.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.312+0000 7f68bd600640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 0x7f68b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.313+0000 7f68bcc00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.313+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac00cd80 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.313+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b819f750 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.313+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac010910 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.313+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0188f0 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.314+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a7030 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.314+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8101e20 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.317+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68ac010ac0 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.318+0000 7f68b5600640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 0x7f68a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.318+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f68ac0cf400 con 0x7f68b81019c0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.318+0000 7f68b7e00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 0x7f68a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.319+0000 7f68b7e00640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 0x7f68a00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68b810e570 tx=0x7f68a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:33.324 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.320+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68ac09c760 con 0x7f68b81019c0 2024-09-12T03:02:33.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:33.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6836/3595237430,v1:172.21.15.57:6837/3595237430] up:boot 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: fsmap 4 up:standby 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:33.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:33 smithi181 ceph-mon[35320]: Deploying daemon mds.e on smithi181 2024-09-12T03:02:33.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.544+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68b819b810 con 0x7f68b81019c0 2024-09-12T03:02:33.545 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:33.545 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6836/3595237430,v1:172.21.15.57:6837/3595237430] up:boot 2024-09-12T03:02:33.545 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: fsmap 4 up:standby 2024-09-12T03:02:33.545 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:33.546 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:33 smithi057 ceph-mon[30361]: Deploying daemon mds.e on smithi181 2024-09-12T03:02:33.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.545+0000 7f68b5600640 1 -- 172.21.15.57:0/1106639073 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7f68ac09c060 con 0x7f68b81019c0 2024-09-12T03:02:33.547 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:33.547 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":5,"btime":"2024-09-12T03:02:32:304841+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-12T03:02:33.547 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 5 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 msgr2=0x7f68a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 0x7f68a00a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68b810e570 tx=0x7f68a4005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 msgr2=0x7f68b819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00e9f0 tx=0x7f68ac00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 shutdown_connections 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f68a00a2b80 0x7f68a00a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b81079c0 0x7f68b819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b8102360 0x7f68b819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 --2- 172.21.15.57:0/1106639073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f68b81019c0 0x7f68b819a4b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:33.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 >> 172.21.15.57:0/1106639073 conn(0x7f68b80fd740 msgr2=0x7f68b810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:33.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 shutdown_connections 2024-09-12T03:02:33.549 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:33.547+0000 7f68be000640 1 -- 172.21.15.57:0/1106639073 wait complete. 2024-09-12T03:02:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6836/3595237430,v1:172.21.15.57:6837/3595237430] up:boot 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: fsmap 4 up:standby 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:33.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:33 smithi204 ceph-mon[35013]: Deploying daemon mds.e on smithi181 2024-09-12T03:02:34.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:34 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1106639073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:34.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:34 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1106639073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:34.682 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:34 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1106639073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:34.683 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:34 smithi181 systemd[1]: Starting Ceph mds.e for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:34.953 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:34 smithi181 podman[64788]: 2024-09-12 03:02:34.585436079 +0000 UTC m=+0.015978035 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:34.953 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:34 smithi181 podman[64788]: 2024-09-12 03:02:34.717749152 +0000 UTC m=+0.148291148 container create bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:34 smithi181 podman[64788]: 2024-09-12 03:02:34.98731545 +0000 UTC m=+0.417857398 container init bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:34 smithi181 podman[64788]: 2024-09-12 03:02:34.990184148 +0000 UTC m=+0.420726098 container start bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mds[64807]: -- 172.21.15.181:0/1312033367 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f74c45860 con 0x556f74c22800 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e[64803]: starting mds.e at 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:35 smithi181 bash[64788]: bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 2024-09-12T03:02:35.341 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:02:35 smithi181 systemd[1]: Started Ceph mds.e for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:35 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:35.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:35.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:35 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:35 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:36.206 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:36.594 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:36.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:36 smithi057 ceph-mon[30361]: Deploying daemon mds.f on smithi204 2024-09-12T03:02:36.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:36 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6834/1691601828,v1:172.21.15.181:6835/1691601828] up:boot 2024-09-12T03:02:36.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:36 smithi057 ceph-mon[30361]: fsmap 5 up:standby 2024-09-12T03:02:36.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:36 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:02:36.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:36 smithi204 ceph-mon[35013]: Deploying daemon mds.f on smithi204 2024-09-12T03:02:36.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:36 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6834/1691601828,v1:172.21.15.181:6835/1691601828] up:boot 2024-09-12T03:02:36.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:36 smithi204 ceph-mon[35013]: fsmap 5 up:standby 2024-09-12T03:02:36.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:36 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:02:36.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:36 smithi181 ceph-mon[35320]: Deploying daemon mds.f on smithi204 2024-09-12T03:02:36.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:36 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6834/1691601828,v1:172.21.15.181:6835/1691601828] up:boot 2024-09-12T03:02:36.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:36 smithi181 ceph-mon[35320]: fsmap 5 up:standby 2024-09-12T03:02:36.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:36 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.150+0000 7f9054400640 1 -- 172.21.15.57:0/3950235367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c102790 msgr2=0x7f904c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.150+0000 7f9054400640 1 --2- 172.21.15.57:0/3950235367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c102790 0x7f904c1067e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9038009900 tx=0x7f903802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 -- 172.21.15.57:0/3950235367 shutdown_connections 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 --2- 172.21.15.57:0/3950235367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c106f10 0x7f904c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 --2- 172.21.15.57:0/3950235367 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c102790 0x7f904c1067e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 --2- 172.21.15.57:0/3950235367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 -- 172.21.15.57:0/3950235367 >> 172.21.15.57:0/3950235367 conn(0x7f904c0fd8c0 msgr2=0x7f904c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 -- 172.21.15.57:0/3950235367 shutdown_connections 2024-09-12T03:02:37.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.151+0000 7f9054400640 1 -- 172.21.15.57:0/3950235367 wait complete. 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.152+0000 7f9054400640 1 Processor -- start 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.152+0000 7f9054400640 1 -- start start 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c102790 0x7f904c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c106f10 0x7f904c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f904c111fc0 con 0x7f904c106f10 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f904c111e40 con 0x7f904c102790 2024-09-12T03:02:37.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9054400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f904c112140 con 0x7f904c101de0 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9052400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c106f10 0x7f904c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60910/0 (socket says 172.21.15.57:60910) 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 -- 172.21.15.57:0/2493199451 learned_addr learned my addr 172.21.15.57:0/2493199451 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051000640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c102790 0x7f904c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 -- 172.21.15.57:0/2493199451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c102790 msgr2=0x7f904c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c102790 0x7f904c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 -- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c106f10 msgr2=0x7f904c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c106f10 0x7f904c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.153+0000 7f9051a00640 1 -- 172.21.15.57:0/2493199451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c19d3f0 con 0x7f904c101de0 2024-09-12T03:02:37.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f9051a00640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f904000e9f0 tx=0x7f904000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904000cd80 con 0x7f904c101de0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9040005500 con 0x7f904c101de0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c19d6e0 con 0x7f904c101de0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040010690 con 0x7f904c101de0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.154+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c1a4fc0 con 0x7f904c101de0 2024-09-12T03:02:37.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.155+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c1021c0 con 0x7f904c101de0 2024-09-12T03:02:37.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.157+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f90400046c0 con 0x7f904c101de0 2024-09-12T03:02:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.158+0000 7f904a000640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 0x7f90340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.159+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f90400cefb0 con 0x7f904c101de0 2024-09-12T03:02:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.159+0000 7f9051000640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 0x7f90340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:37.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.159+0000 7f9051000640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 0x7f90340a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9038000c00 tx=0x7f9038005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:37.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.160+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f904009c310 con 0x7f904c101de0 2024-09-12T03:02:37.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.383+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f904c1997a0 con 0x7f904c101de0 2024-09-12T03:02:37.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.384+0000 7f904a000640 1 -- 172.21.15.57:0/2493199451 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7f904c1997a0 con 0x7f904c101de0 2024-09-12T03:02:37.385 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:37.385 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":6,"btime":"2024-09-12T03:02:35:335082+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:02:37.385 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 6 2024-09-12T03:02:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 msgr2=0x7f90340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 0x7f90340a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9038000c00 tx=0x7f9038005b50 comp rx=0 tx=0).stop 2024-09-12T03:02:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 msgr2=0x7f904c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:37.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f904000e9f0 tx=0x7f904000edf0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 shutdown_connections 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90340a2c00 0x7f90340a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f904c106f10 0x7f904c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f904c102790 0x7f904c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 --2- 172.21.15.57:0/2493199451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f904c101de0 0x7f904c198440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.386+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 >> 172.21.15.57:0/2493199451 conn(0x7f904c0fd8c0 msgr2=0x7f904c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.387+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 shutdown_connections 2024-09-12T03:02:37.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:37.387+0000 7f9054400640 1 -- 172.21.15.57:0/2493199451 wait complete. 2024-09-12T03:02:37.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:37 smithi204 ceph-mon[35013]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:37.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:37 smithi057 ceph-mon[30361]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:37 smithi181 ceph-mon[35320]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:37.816 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:02:37 smithi204 podman[65119]: 2024-09-12 03:02:37.792783762 +0000 UTC m=+0.361418775 container start 0ece4f5feaf126fa82b98269f47095994019184c4fd2dae763df2e2c6b904d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:02:38.143 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:02:37 smithi204 ceph-mds[65138]: -- 172.21.15.204:0/1155822836 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f790a9860 con 0x558f79087000 2024-09-12T03:02:38.143 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:02:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: starting mds.f at 2024-09-12T03:02:38.144 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:02:37 smithi204 bash[65119]: 0ece4f5feaf126fa82b98269f47095994019184c4fd2dae763df2e2c6b904d2f 2024-09-12T03:02:38.144 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:02:37 smithi204 systemd[1]: Started Ceph mds.f for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2493199451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:38.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:38 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2493199451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:38.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:38 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:38.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2493199451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:38.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:38.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:38.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:38.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:38 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:39.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:39 smithi204 ceph-mon[35013]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:39.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:39 smithi204 ceph-mon[35013]: Deploying daemon mds.c on smithi204 2024-09-12T03:02:39.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:39 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6834/1804109241,v1:172.21.15.204:6835/1804109241] up:boot 2024-09-12T03:02:39.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:39 smithi204 ceph-mon[35013]: fsmap 6 up:standby 2024-09-12T03:02:39.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:39 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:02:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:39 smithi057 ceph-mon[30361]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:39 smithi057 ceph-mon[30361]: Deploying daemon mds.c on smithi204 2024-09-12T03:02:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:39 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6834/1804109241,v1:172.21.15.204:6835/1804109241] up:boot 2024-09-12T03:02:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:39 smithi057 ceph-mon[30361]: fsmap 6 up:standby 2024-09-12T03:02:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:39 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:02:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:39 smithi181 ceph-mon[35320]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:39 smithi181 ceph-mon[35320]: Deploying daemon mds.c on smithi204 2024-09-12T03:02:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:39 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6834/1804109241,v1:172.21.15.204:6835/1804109241] up:boot 2024-09-12T03:02:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:39 smithi181 ceph-mon[35320]: fsmap 6 up:standby 2024-09-12T03:02:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:39 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:02:40.143 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 systemd[1]: Starting Ceph mds.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:40.315 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:40.556 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 podman[65419]: 2024-09-12 03:02:40.172377596 +0000 UTC m=+0.016264135 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:40.556 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 podman[65419]: 2024-09-12 03:02:40.287117373 +0000 UTC m=+0.131003968 container create 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:02:40.556 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 podman[65419]: 2024-09-12 03:02:40.556521525 +0000 UTC m=+0.400408051 container init 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:02:40.702 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:40.893 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 podman[65419]: 2024-09-12 03:02:40.559425594 +0000 UTC m=+0.403312130 container start 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T03:02:40.893 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 ceph-mds[65439]: -- 172.21.15.204:0/2000854286 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5581e675f860 con 0x5581e673c800 2024-09-12T03:02:40.893 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: starting mds.c at 2024-09-12T03:02:40.894 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 bash[65419]: 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 2024-09-12T03:02:40.894 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:02:40 smithi204 systemd[1]: Started Ceph mds.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:41.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 -- 172.21.15.57:0/753780247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8102360 msgr2=0x7ff2c810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 --2- 172.21.15.57:0/753780247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8102360 0x7ff2c810e890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009be0 tx=0x7ff2c002a6d0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 -- 172.21.15.57:0/753780247 shutdown_connections 2024-09-12T03:02:41.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 --2- 172.21.15.57:0/753780247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8102360 0x7ff2c810e890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 --2- 172.21.15.57:0/753780247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c81019c0 0x7ff2c8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 --2- 172.21.15.57:0/753780247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 0x7ff2c8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 -- 172.21.15.57:0/753780247 >> 172.21.15.57:0/753780247 conn(0x7ff2c80fd740 msgr2=0x7ff2c80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 -- 172.21.15.57:0/753780247 shutdown_connections 2024-09-12T03:02:41.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.259+0000 7ff2cce00640 1 -- 172.21.15.57:0/753780247 wait complete. 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.260+0000 7ff2cce00640 1 Processor -- start 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.260+0000 7ff2cce00640 1 -- start start 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 0x7ff2c819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 0x7ff2c819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111d70 con 0x7ff2c81079c0 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111bf0 con 0x7ff2c81019c0 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2cce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111ef0 con 0x7ff2c8102360 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2c7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 0x7ff2c819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:44174/0 (socket says 172.21.15.57:44174) 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 -- 172.21.15.57:0/283190317 learned_addr learned my addr 172.21.15.57:0/283190317 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2c7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 0x7ff2c819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 -- 172.21.15.57:0/283190317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 msgr2=0x7ff2c819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 0x7ff2c819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 -- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 msgr2=0x7ff2c819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 0x7ff2c819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.261+0000 7ff2bfe00640 1 -- 172.21.15.57:0/283190317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c819f410 con 0x7ff2c8102360 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2c7400640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 0x7ff2c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2bfe00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9f0 tx=0x7ff2b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000ccc0 con 0x7ff2c8102360 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b000ce60 con 0x7ff2c8102360 2024-09-12T03:02:41.264 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c819f700 con 0x7ff2c8102360 2024-09-12T03:02:41.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0010690 con 0x7ff2c8102360 2024-09-12T03:02:41.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.262+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a6fe0 con 0x7ff2c8102360 2024-09-12T03:02:41.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.263+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8101e20 con 0x7ff2c8102360 2024-09-12T03:02:41.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.265+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff2b00046c0 con 0x7ff2c8102360 2024-09-12T03:02:41.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.266+0000 7ff2c4c00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 0x7ff2ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:41.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.266+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff2b00cf060 con 0x7ff2c8102360 2024-09-12T03:02:41.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.266+0000 7ff2c7400640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 0x7ff2ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:41.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.267+0000 7ff2c7400640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 0x7ff2ac0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005fd0 tx=0x7ff2b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:41.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.270+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b009c3c0 con 0x7ff2c8102360 2024-09-12T03:02:41.440 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:41.441 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:41.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.496+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c81a72c0 con 0x7ff2c8102360 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:41.497 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:41.498 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:41.498 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.496+0000 7ff2c4c00640 1 -- 172.21.15.57:0/283190317 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7ff2b009bcc0 con 0x7ff2c8102360 2024-09-12T03:02:41.498 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:41.498 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":8,"btime":"2024-09-12T03:02:41:361515+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:02:41.499 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 8 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 msgr2=0x7ff2ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 0x7ff2ac0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005fd0 tx=0x7ff2b8009260 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 msgr2=0x7ff2c819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000e9f0 tx=0x7ff2b000edf0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 shutdown_connections 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2ac0a2b80 0x7ff2ac0a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff2c81079c0 0x7ff2c819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2c8102360 0x7ff2c819a9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 --2- 172.21.15.57:0/283190317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c81019c0 0x7ff2c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:41.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 >> 172.21.15.57:0/283190317 conn(0x7ff2c80fd740 msgr2=0x7ff2c810cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:41.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 shutdown_connections 2024-09-12T03:02:41.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:41.499+0000 7ff2cce00640 1 -- 172.21.15.57:0/283190317 wait complete. 2024-09-12T03:02:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:41.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:42 smithi181 ceph-mon[35320]: Deploying daemon mds.h on smithi181 2024-09-12T03:02:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:42 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6836/368351857,v1:172.21.15.204:6837/368351857] up:boot 2024-09-12T03:02:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:42 smithi181 ceph-mon[35320]: fsmap 7 up:standby 2024-09-12T03:02:42.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:02:42.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:42 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/283190317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:42 smithi057 ceph-mon[30361]: Deploying daemon mds.h on smithi181 2024-09-12T03:02:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:42 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6836/368351857,v1:172.21.15.204:6837/368351857] up:boot 2024-09-12T03:02:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:42 smithi057 ceph-mon[30361]: fsmap 7 up:standby 2024-09-12T03:02:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:02:42.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:42 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/283190317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:42 smithi204 ceph-mon[35013]: Deploying daemon mds.h on smithi181 2024-09-12T03:02:42.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:42 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6836/368351857,v1:172.21.15.204:6837/368351857] up:boot 2024-09-12T03:02:42.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:42 smithi204 ceph-mon[35013]: fsmap 7 up:standby 2024-09-12T03:02:42.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:02:42.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:42 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/283190317' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.953 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:42 smithi181 systemd[1]: Starting Ceph mds.h for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:43.294 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 podman[65088]: 2024-09-12 03:02:42.917122877 +0000 UTC m=+0.015861742 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:43.354 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 podman[65088]: 2024-09-12 03:02:43.040533974 +0000 UTC m=+0.139272835 container create d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:02:43.355 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 podman[65088]: 2024-09-12 03:02:43.293417652 +0000 UTC m=+0.392156513 container init d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:02:43.603 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:43 smithi057 ceph-mon[30361]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:43.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:43 smithi204 ceph-mon[35013]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:43 smithi181 ceph-mon[35320]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:43.703 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 podman[65088]: 2024-09-12 03:02:43.296340756 +0000 UTC m=+0.395079617 container start d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:02:43.703 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 ceph-mds[65107]: -- 172.21.15.181:0/3362391907 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b42bf07860 con 0x55b42bee5000 2024-09-12T03:02:43.703 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: starting mds.h at 2024-09-12T03:02:43.703 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 bash[65088]: d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 2024-09-12T03:02:43.704 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:02:43 smithi181 systemd[1]: Started Ceph mds.h for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:44.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:44.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6836/2458916124,v1:172.21.15.181:6837/2458916124] up:boot 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: fsmap 8 up:standby 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:44 smithi057 ceph-mon[30361]: Deploying daemon mds.i on smithi204 2024-09-12T03:02:44.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6836/2458916124,v1:172.21.15.181:6837/2458916124] up:boot 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: fsmap 8 up:standby 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:44.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:44 smithi204 ceph-mon[35013]: Deploying daemon mds.i on smithi204 2024-09-12T03:02:44.645 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6836/2458916124,v1:172.21.15.181:6837/2458916124] up:boot 2024-09-12T03:02:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: fsmap 8 up:standby 2024-09-12T03:02:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:02:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:44.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:44.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:44.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:44.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:44 smithi181 ceph-mon[35320]: Deploying daemon mds.i on smithi204 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 -- 172.21.15.57:0/639215027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4104010 msgr2=0x7fefc41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 --2- 172.21.15.57:0/639215027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4104010 0x7fefc41043f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009900 tx=0x7fefb402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 -- 172.21.15.57:0/639215027 shutdown_connections 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 --2- 172.21.15.57:0/639215027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc4109140 0x7fefc4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 --2- 172.21.15.57:0/639215027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc41049c0 0x7fefc4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 --2- 172.21.15.57:0/639215027 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4104010 0x7fefc41043f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 -- 172.21.15.57:0/639215027 >> 172.21.15.57:0/639215027 conn(0x7fefc40ffa50 msgr2=0x7fefc4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 -- 172.21.15.57:0/639215027 shutdown_connections 2024-09-12T03:02:45.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.201+0000 7fefc8c00640 1 -- 172.21.15.57:0/639215027 wait complete. 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.202+0000 7fefc8c00640 1 Processor -- start 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.202+0000 7fefc8c00640 1 -- start start 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.202+0000 7fefc8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.202+0000 7fefc8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41049c0 0x7fefc419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.202+0000 7fefc8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 0x7fefc419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc8c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4114190 con 0x7fefc4109140 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4114010 con 0x7fefc4104010 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4114310 con 0x7fefc41049c0 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefbbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41049c0 0x7fefc419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 0x7fefc419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58086/0 (socket says 172.21.15.57:58086) 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 -- 172.21.15.57:0/1969309557 learned_addr learned my addr 172.21.15.57:0/1969309557 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 -- 172.21.15.57:0/1969309557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41049c0 msgr2=0x7fefc419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41049c0 0x7fefc419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 -- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 msgr2=0x7fefc419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 0x7fefc419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc1e00640 1 -- 172.21.15.57:0/1969309557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc419f680 con 0x7fefc4104010 2024-09-12T03:02:45.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.203+0000 7fefc2800640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 0x7fefc419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:45.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefc1e00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefb4004770 tx=0x7fefb4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb403f070 con 0x7fefc4104010 2024-09-12T03:02:45.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc419f910 con 0x7fefc4104010 2024-09-12T03:02:45.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb4031ca0 con 0x7fefc4104010 2024-09-12T03:02:45.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4035030 con 0x7fefc4104010 2024-09-12T03:02:45.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.204+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a7250 con 0x7fefc4104010 2024-09-12T03:02:45.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.205+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc41043f0 con 0x7fefc4104010 2024-09-12T03:02:45.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.207+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefb4042070 con 0x7fefc4104010 2024-09-12T03:02:45.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.208+0000 7fefbaa00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 0x7fefa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.209+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fefb40ef480 con 0x7fefc4104010 2024-09-12T03:02:45.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.209+0000 7fefbbe00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 0x7fefa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.209+0000 7fefbbe00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 0x7fefa80a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefc40ffe70 tx=0x7fefac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.211 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.210+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefb40bc190 con 0x7fefc4104010 2024-09-12T03:02:45.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.435+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefc410c320 con 0x7fefc4104010 2024-09-12T03:02:45.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.436+0000 7fefbaa00640 1 -- 172.21.15.57:0/1969309557 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7fefb40c0490 con 0x7fefc4104010 2024-09-12T03:02:45.437 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:45.437 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":9,"btime":"2024-09-12T03:02:43:377000+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-12T03:02:45.438 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 9 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 msgr2=0x7fefa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 0x7fefa80a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefc40ffe70 tx=0x7fefac00b410 comp rx=0 tx=0).stop 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 msgr2=0x7fefc419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fefb4004770 tx=0x7fefb4003c10 comp rx=0 tx=0).stop 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 shutdown_connections 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fefa80a2b80 0x7fefa80a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fefc4109140 0x7fefc419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41049c0 0x7fefc419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 --2- 172.21.15.57:0/1969309557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefc4104010 0x7fefc419a6d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 >> 172.21.15.57:0/1969309557 conn(0x7fefc40ffa50 msgr2=0x7fefc4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 shutdown_connections 2024-09-12T03:02:45.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:45.439+0000 7fefc8c00640 1 -- 172.21.15.57:0/1969309557 wait complete. 2024-09-12T03:02:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:45 smithi204 ceph-mon[35013]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:45 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1969309557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:45.644 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:45 smithi204 systemd[1]: Starting Ceph mds.i for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:45.797 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:45 smithi057 ceph-mon[30361]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:45.798 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:45 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1969309557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:45 smithi181 ceph-mon[35320]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:45 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1969309557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:46.047 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:45 smithi204 podman[65723]: 2024-09-12 03:02:45.660454694 +0000 UTC m=+0.016152474 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:46.047 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:45 smithi204 podman[65723]: 2024-09-12 03:02:45.777698432 +0000 UTC m=+0.133396274 container create 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:02:46.393 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 podman[65723]: 2024-09-12 03:02:46.047452811 +0000 UTC m=+0.403150595 container init 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:02:46.394 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 podman[65723]: 2024-09-12 03:02:46.050263231 +0000 UTC m=+0.405961012 container start 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:02:46.394 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 ceph-mds[65742]: -- 172.21.15.204:0/4292463927 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564953cbc20 con 0x5564953a8800 2024-09-12T03:02:46.394 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: starting mds.i at 2024-09-12T03:02:46.394 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 bash[65723]: 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 2024-09-12T03:02:46.394 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:02:46 smithi204 systemd[1]: Started Ceph mds.i for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:47.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: Deploying daemon mds.g on smithi057 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:boot 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: fsmap 9 up:standby 2024-09-12T03:02:47.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:47 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: Deploying daemon mds.g on smithi057 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:boot 2024-09-12T03:02:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: fsmap 9 up:standby 2024-09-12T03:02:47.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:47 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:02:47.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:47.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:47.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: Deploying daemon mds.g on smithi057 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:boot 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: fsmap 9 up:standby 2024-09-12T03:02:47.645 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:47 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:02:48.220 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:48.677 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:49.247 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:49 smithi057 ceph-mon[30361]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 -- 172.21.15.57:0/2453913094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 msgr2=0x7f5b681089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 --2- 172.21.15.57:0/2453913094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b681089f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009900 tx=0x7f5b5402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 -- 172.21.15.57:0/2453913094 shutdown_connections 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 --2- 172.21.15.57:0/2453913094 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68109120 0x7f5b68110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 --2- 172.21.15.57:0/2453913094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b681089f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 --2- 172.21.15.57:0/2453913094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68103ff0 0x7f5b681043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 -- 172.21.15.57:0/2453913094 >> 172.21.15.57:0/2453913094 conn(0x7f5b680ffa90 msgr2=0x7f5b68101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 -- 172.21.15.57:0/2453913094 shutdown_connections 2024-09-12T03:02:49.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.261+0000 7f5b6e000640 1 -- 172.21.15.57:0/2453913094 wait complete. 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.262+0000 7f5b6e000640 1 Processor -- start 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 -- start start 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b6819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68109120 0x7f5b6819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68114140 con 0x7f5b68103ff0 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68113fc0 con 0x7f5b68109120 2024-09-12T03:02:49.265 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b6e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681142c0 con 0x7f5b681049a0 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b67400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40702/0 (socket says 172.21.15.57:40702) 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b67400640 1 -- 172.21.15.57:0/4167827041 learned_addr learned my addr 172.21.15.57:0/4167827041 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b66a00640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b6819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.263+0000 7f5b67e00640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68109120 0x7f5b6819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 -- 172.21.15.57:0/4167827041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 msgr2=0x7f5b6819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b6819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 -- 172.21.15.57:0/4167827041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68109120 msgr2=0x7f5b6819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68109120 0x7f5b6819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 -- 172.21.15.57:0/4167827041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819f5d0 con 0x7f5b68103ff0 2024-09-12T03:02:49.266 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b67400640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9f0 tx=0x7f5b5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800cd80 con 0x7f5b68103ff0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b58005500 con 0x7f5b68103ff0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58010690 con 0x7f5b68103ff0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.264+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6819f8c0 con 0x7f5b68103ff0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.265+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a71a0 con 0x7f5b68103ff0 2024-09-12T03:02:49.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.266+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b681043d0 con 0x7f5b68103ff0 2024-09-12T03:02:49.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.268+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5b580046c0 con 0x7f5b68103ff0 2024-09-12T03:02:49.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.270+0000 7f5b5fe00640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 0x7f5b4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.270+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f5b580cefc0 con 0x7f5b68103ff0 2024-09-12T03:02:49.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.270+0000 7f5b66a00640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 0x7f5b4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.271+0000 7f5b66a00640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 0x7f5b4c0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009230 tx=0x7f5b540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.274+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b5809c320 con 0x7f5b68103ff0 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.499+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b6810c390 con 0x7f5b68103ff0 2024-09-12T03:02:49.500 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:49 smithi057 podman[70813]: 2024-09-12 03:02:49.313578704 +0000 UTC m=+0.027762539 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.500+0000 7f5b5fe00640 1 -- 172.21.15.57:0/4167827041 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7f5b58086070 con 0x7f5b68103ff0 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":10,"btime":"2024-09-12T03:02:46:469777+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 10 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 msgr2=0x7f5b4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 0x7f5b4c0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009230 tx=0x7f5b540058d0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 msgr2=0x7f5b6819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e9f0 tx=0x7f5b5800edf0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 shutdown_connections 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5b4c0a2c00 0x7f5b4c0a50c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b68109120 0x7f5b6819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b681049a0 0x7f5b6819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 --2- 172.21.15.57:0/4167827041 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5b68103ff0 0x7f5b6819a620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 >> 172.21.15.57:0/4167827041 conn(0x7f5b680ffa90 msgr2=0x7f5b68100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 shutdown_connections 2024-09-12T03:02:49.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:49.502+0000 7f5b6e000640 1 -- 172.21.15.57:0/4167827041 wait complete. 2024-09-12T03:02:49.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:49 smithi204 ceph-mon[35013]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:49 smithi181 ceph-mon[35320]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:49.869 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:49 smithi057 podman[70813]: 2024-09-12 03:02:49.550304855 +0000 UTC m=+0.264488686 container create aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:02:50.547 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:50 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4167827041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:50.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:50 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/4167827041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:50.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:50 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/4167827041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 podman[70813]: 2024-09-12 03:02:50.547692501 +0000 UTC m=+1.261876336 container init aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 podman[70813]: 2024-09-12 03:02:50.553793436 +0000 UTC m=+1.267977271 container start aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 ceph-mds[70849]: -- 172.21.15.57:0/4265663135 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a66fed860 con 0x559a66fca800 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g[70845]: starting mds.g at 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 bash[70813]: aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f 2024-09-12T03:02:50.870 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:02:50 smithi057 systemd[1]: Started Ceph mds.g for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:51.505 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:51 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:51.605 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:51.605 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.605 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.605 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.605 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:51.606 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:51.606 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:51 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:51.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:51.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:51.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:51 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:52.252 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:52.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:52 smithi057 ceph-mon[30361]: Deploying daemon mds.a on smithi057 2024-09-12T03:02:52.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:52 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6838/1375693587,v1:172.21.15.57:6839/1375693587] up:boot 2024-09-12T03:02:52.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:52 smithi057 ceph-mon[30361]: fsmap 10 up:standby 2024-09-12T03:02:52.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:52 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:02:52.627 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:52 smithi181 ceph-mon[35320]: Deploying daemon mds.a on smithi057 2024-09-12T03:02:52.627 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:52 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6838/1375693587,v1:172.21.15.57:6839/1375693587] up:boot 2024-09-12T03:02:52.628 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:52 smithi181 ceph-mon[35320]: fsmap 10 up:standby 2024-09-12T03:02:52.628 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:52 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:02:52.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:52 smithi204 ceph-mon[35013]: Deploying daemon mds.a on smithi057 2024-09-12T03:02:52.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:52 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6838/1375693587,v1:172.21.15.57:6839/1375693587] up:boot 2024-09-12T03:02:52.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:52 smithi204 ceph-mon[35013]: fsmap 10 up:standby 2024-09-12T03:02:52.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:52 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:02:52.668 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:53.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 -- 172.21.15.57:0/2063238423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c10e280 msgr2=0x7fcd1c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.477 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 --2- 172.21.15.57:0/2063238423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c10e280 0x7fcd1c110670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009ca0 tx=0x7fcd1402a950 comp rx=0 tx=0).stop 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 -- 172.21.15.57:0/2063238423 shutdown_connections 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 --2- 172.21.15.57:0/2063238423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c10e280 0x7fcd1c110670 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 --2- 172.21.15.57:0/2063238423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 0x7fcd1c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 --2- 172.21.15.57:0/2063238423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c074510 0x7fcd1c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 -- 172.21.15.57:0/2063238423 >> 172.21.15.57:0/2063238423 conn(0x7fcd1c0fd740 msgr2=0x7fcd1c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 -- 172.21.15.57:0/2063238423 shutdown_connections 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.476+0000 7fcd22e00640 1 -- 172.21.15.57:0/2063238423 wait complete. 2024-09-12T03:02:53.478 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.477+0000 7fcd22e00640 1 Processor -- start 2024-09-12T03:02:53.479 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.477+0000 7fcd22e00640 1 -- start start 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 0x7fcd1c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 0x7fcd1c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 0x7fcd1c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c113fb0 con 0x7fcd1c074510 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c113e30 con 0x7fcd1c10e280 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c114130 con 0x7fcd1c074e30 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd21a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 0x7fcd1c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 0x7fcd1c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd21a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 0x7fcd1c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40754/0 (socket says 172.21.15.57:40754) 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd21a00640 1 -- 172.21.15.57:0/1712860329 learned_addr learned my addr 172.21.15.57:0/1712860329 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd21000640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 0x7fcd1c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.480 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22400640 1 -- 172.21.15.57:0/1712860329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 msgr2=0x7fcd1c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22400640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 0x7fcd1c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22400640 1 -- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 msgr2=0x7fcd1c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.478+0000 7fcd22400640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 0x7fcd1c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd22400640 1 -- 172.21.15.57:0/1712860329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c19f460 con 0x7fcd1c10e280 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd22400640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 0x7fcd1c19ed80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402ae70 tx=0x7fcd1402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14034040 con 0x7fcd1c10e280 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c19f6f0 con 0x7fcd1c10e280 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd14030920 con 0x7fcd1c10e280 2024-09-12T03:02:53.481 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140045b0 con 0x7fcd1c10e280 2024-09-12T03:02:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.479+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c1a7030 con 0x7fcd1c10e280 2024-09-12T03:02:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.480+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c0748f0 con 0x7fcd1c10e280 2024-09-12T03:02:53.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.482+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcd14037020 con 0x7fcd1c10e280 2024-09-12T03:02:53.483 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:53 smithi057 ceph-mon[30361]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:53.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.484+0000 7fcd1a000640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:53.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.485+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fcd140ef8c0 con 0x7fcd1c10e280 2024-09-12T03:02:53.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.485+0000 7fcd21a00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:53.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.485+0000 7fcd21a00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 0x7fcd040a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009d10 tx=0x7fcd10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:53.486 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.486+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd140bcc20 con 0x7fcd1c10e280 2024-09-12T03:02:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:53 smithi204 ceph-mon[35013]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:53 smithi181 ceph-mon[35320]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:53.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.722+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd1c109400 con 0x7fcd1c10e280 2024-09-12T03:02:53.723 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.723+0000 7fcd1a000640 1 -- 172.21.15.57:0/1712860329 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7fcd140098b0 con 0x7fcd1c10e280 2024-09-12T03:02:53.724 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:53.724 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":11,"btime":"2024-09-12T03:02:51:219289+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:02:53.724 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 11 2024-09-12T03:02:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 msgr2=0x7fcd040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 0x7fcd040a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009d10 tx=0x7fcd10005f90 comp rx=0 tx=0).stop 2024-09-12T03:02:53.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 msgr2=0x7fcd1c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 0x7fcd1c19ed80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402ae70 tx=0x7fcd1402f3a0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 shutdown_connections 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c10e280 0x7fcd1c19ed80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd1c074e30 0x7fcd1c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 --2- 172.21.15.57:0/1712860329 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcd1c074510 0x7fcd1c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 >> 172.21.15.57:0/1712860329 conn(0x7fcd1c0fd740 msgr2=0x7fcd1c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 shutdown_connections 2024-09-12T03:02:53.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:53.726+0000 7fcd22e00640 1 -- 172.21.15.57:0/1712860329 wait complete. 2024-09-12T03:02:54.459 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:54 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1712860329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:54.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:54 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1712860329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:54.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:54 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1712860329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: Deploying daemon mds.b on smithi181 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:boot 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: fsmap 11 up:standby 2024-09-12T03:02:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:55 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:55.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:55.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: Deploying daemon mds.b on smithi181 2024-09-12T03:02:55.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:boot 2024-09-12T03:02:55.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: fsmap 11 up:standby 2024-09-12T03:02:55.786 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:55 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:02:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: Deploying daemon mds.b on smithi181 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:boot 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: fsmap 11 up:standby 2024-09-12T03:02:55.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:55 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:02:56.453 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 systemd[1]: Starting Ceph mds.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:02:56.821 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 podman[65399]: 2024-09-12 03:02:56.458023279 +0000 UTC m=+0.015795642 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:02:56.821 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 podman[65399]: 2024-09-12 03:02:56.569187178 +0000 UTC m=+0.126959531 container create e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:02:56.886 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 podman[65399]: 2024-09-12 03:02:56.821503869 +0000 UTC m=+0.379276223 container init e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 podman[65399]: 2024-09-12 03:02:56.824608421 +0000 UTC m=+0.382380772 container start e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 ceph-mds[65419]: -- 172.21.15.181:0/3903631998 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556eb22f860 con 0x5556eb20d400 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: starting mds.b at 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 bash[65399]: e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e 2024-09-12T03:02:57.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:02:56 smithi181 systemd[1]: Started Ceph mds.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:02:57.281 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:57.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:57.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.659 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.660 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:57 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:57 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:57.845 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:57 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.885+0000 7f9977800640 1 -- 172.21.15.57:0/1423747185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9970103ff0 msgr2=0x7f99701043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.885+0000 7f9977800640 1 --2- 172.21.15.57:0/1423747185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9970103ff0 0x7f99701043d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9964009860 tx=0x7f996402a340 comp rx=0 tx=0).stop 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 -- 172.21.15.57:0/1423747185 shutdown_connections 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 --2- 172.21.15.57:0/1423747185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970109120 0x7f9970110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 --2- 172.21.15.57:0/1423747185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f99701049a0 0x7f99701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 --2- 172.21.15.57:0/1423747185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9970103ff0 0x7f99701043d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 -- 172.21.15.57:0/1423747185 >> 172.21.15.57:0/1423747185 conn(0x7f99700ffa90 msgr2=0x7f9970101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 -- 172.21.15.57:0/1423747185 shutdown_connections 2024-09-12T03:02:57.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.886+0000 7f9977800640 1 -- 172.21.15.57:0/1423747185 wait complete. 2024-09-12T03:02:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 Processor -- start 2024-09-12T03:02:57.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 -- start start 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 0x7f997019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9970109120 0x7f997019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9970114150 con 0x7f9970109120 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9970113fd0 con 0x7f99701049a0 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.887+0000 7f9977800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99701142d0 con 0x7f9970103ff0 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:53304/0 (socket says 172.21.15.57:53304) 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 -- 172.21.15.57:0/1911568311 learned_addr learned my addr 172.21.15.57:0/1911568311 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f996fe00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 0x7f997019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9975800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9970109120 0x7f997019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 -- 172.21.15.57:0/1911568311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 msgr2=0x7f997019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 0x7f997019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 -- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9970109120 msgr2=0x7f997019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9970109120 0x7f997019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 -- 172.21.15.57:0/1911568311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997019f620 con 0x7f9970103ff0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f996fe00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 0x7f997019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.888+0000 7f9974e00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9964035040 tx=0x7f9964033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.889+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964037040 con 0x7f9970103ff0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.889+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9964040070 con 0x7f9970103ff0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.889+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640039b0 con 0x7f9970103ff0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.889+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997019f8b0 con 0x7f9970103ff0 2024-09-12T03:02:57.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.889+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701a71f0 con 0x7f9970103ff0 2024-09-12T03:02:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.890+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99701043d0 con 0x7f9970103ff0 2024-09-12T03:02:57.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.892+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f99640305e0 con 0x7f9970103ff0 2024-09-12T03:02:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.893+0000 7f996d600640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 0x7f99580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.893+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f99640eecd0 con 0x7f9970103ff0 2024-09-12T03:02:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.893+0000 7f996fe00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 0x7f99580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:57.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.894+0000 7f996fe00640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 0x7f99580a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99700fff40 tx=0x7f995c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:57.897 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:57.897+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99640bc030 con 0x7f9970103ff0 2024-09-12T03:02:58.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.128+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997019b9d0 con 0x7f9970103ff0 2024-09-12T03:02:58.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.129+0000 7f996d600640 1 -- 172.21.15.57:0/1911568311 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f99640bb930 con 0x7f9970103ff0 2024-09-12T03:02:58.130 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:02:58.130 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":13,"btime":"2024-09-12T03:02:57:397922+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T03:02:58.130 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 13 2024-09-12T03:02:58.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 msgr2=0x7f99580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.132 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 0x7f99580a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f99700fff40 tx=0x7f995c009260 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 msgr2=0x7f997019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9964035040 tx=0x7f9964033b60 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 shutdown_connections 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f99580a2c00 0x7f99580a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9970109120 0x7f997019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99701049a0 0x7f997019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 --2- 172.21.15.57:0/1911568311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9970103ff0 0x7f997019a670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 >> 172.21.15.57:0/1911568311 conn(0x7f99700ffa90 msgr2=0x7f9970100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 shutdown_connections 2024-09-12T03:02:58.133 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:02:58.132+0000 7f9977800640 1 -- 172.21.15.57:0/1911568311 wait complete. 2024-09-12T03:02:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:58 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:boot 2024-09-12T03:02:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:58 smithi057 ceph-mon[30361]: fsmap 12 up:standby 2024-09-12T03:02:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:58 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:02:58.676 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:58 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1911568311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:58 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:boot 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:58 smithi181 ceph-mon[35320]: fsmap 12 up:standby 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:58 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:02:58.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:58 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1911568311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:58 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:boot 2024-09-12T03:02:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:58 smithi204 ceph-mon[35013]: fsmap 12 up:standby 2024-09-12T03:02:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:58 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:02:58.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:58 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1911568311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:59.440 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:02:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:59 smithi057 ceph-mon[30361]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.657 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:02:59 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.851 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:59 smithi181 ceph-mon[35320]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:02:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:02:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:02:59 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:59 smithi204 ceph-mon[35013]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:00.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:02:59 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.434+0000 7fcc55400640 1 -- 172.21.15.57:0/832582036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50108f60 msgr2=0x7fcc50110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.434+0000 7fcc55400640 1 --2- 172.21.15.57:0/832582036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50108f60 0x7fcc50110ae0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009ca0 tx=0x7fcc4802a8b0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 -- 172.21.15.57:0/832582036 shutdown_connections 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 --2- 172.21.15.57:0/832582036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50108f60 0x7fcc50110ae0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 --2- 172.21.15.57:0/832582036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 0x7fcc50108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 --2- 172.21.15.57:0/832582036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50103e30 0x7fcc50104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 -- 172.21.15.57:0/832582036 >> 172.21.15.57:0/832582036 conn(0x7fcc500ff8d0 msgr2=0x7fcc50101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 -- 172.21.15.57:0/832582036 shutdown_connections 2024-09-12T03:03:00.436 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.435+0000 7fcc55400640 1 -- 172.21.15.57:0/832582036 wait complete. 2024-09-12T03:03:00.437 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 Processor -- start 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 -- start start 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 0x7fcc5019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50108f60 0x7fcc5019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50113fc0 con 0x7fcc501047e0 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50113e40 con 0x7fcc50108f60 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.436+0000 7fcc55400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50114140 con 0x7fcc50103e30 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:53316/0 (socket says 172.21.15.57:53316) 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 -- 172.21.15.57:0/20451637 learned_addr learned my addr 172.21.15.57:0/20451637 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc47400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 0x7fcc5019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc54a00640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50108f60 0x7fcc5019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 -- 172.21.15.57:0/20451637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50108f60 msgr2=0x7fcc5019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50108f60 0x7fcc5019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 -- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 msgr2=0x7fcc5019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 0x7fcc5019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.437+0000 7fcc4fe00640 1 -- 172.21.15.57:0/20451637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5019f3b0 con 0x7fcc50103e30 2024-09-12T03:03:00.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc4fe00640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e9f0 tx=0x7fcc4000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000cd80 con 0x7fcc50103e30 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5019f6a0 con 0x7fcc50103e30 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40010910 con 0x7fcc50103e30 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40018980 con 0x7fcc50103e30 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.438+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501a6f80 con 0x7fcc50103e30 2024-09-12T03:03:00.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.440+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50104210 con 0x7fcc50103e30 2024-09-12T03:03:00.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.441+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcc40010ac0 con 0x7fcc50103e30 2024-09-12T03:03:00.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.442+0000 7fcc4d600640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 0x7fcc340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.443+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fcc400ceee0 con 0x7fcc50103e30 2024-09-12T03:03:00.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.443+0000 7fcc47400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 0x7fcc340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.443+0000 7fcc47400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 0x7fcc340a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc380095f0 tx=0x7fcc38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.446+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc4009bbf0 con 0x7fcc50103e30 2024-09-12T03:03:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.641+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcc5002d7c0 con 0x7fcc50103e30 2024-09-12T03:03:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.641+0000 7fcc4d600640 1 -- 172.21.15.57:0/20451637 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fcc4009fef0 con 0x7fcc50103e30 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout:[ 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout: "poolnum": 1, 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout: "poolname": ".mgr" 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:03:00.643 INFO:teuthology.orchestra.run.smithi057.stdout:] 2024-09-12T03:03:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 msgr2=0x7fcc340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 0x7fcc340a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcc380095f0 tx=0x7fcc38005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 msgr2=0x7fcc5019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc4000e9f0 tx=0x7fcc4000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 shutdown_connections 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fcc340a2c00 0x7fcc340a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc50108f60 0x7fcc5019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcc501047e0 0x7fcc5019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 --2- 172.21.15.57:0/20451637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc50103e30 0x7fcc5019a400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 >> 172.21.15.57:0/20451637 conn(0x7fcc500ff8d0 msgr2=0x7fcc50100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 shutdown_connections 2024-09-12T03:03:00.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:00.644+0000 7fcc55400640 1 -- 172.21.15.57:0/20451637 wait complete. 2024-09-12T03:03:01.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:00 smithi057 ceph-mon[30361]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:01.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.030 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:00 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.031 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:00 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/20451637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:03:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:00 smithi204 ceph-mon[35013]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:00 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:00 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/20451637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:03:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:00 smithi181 ceph-mon[35320]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:00 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:00 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/20451637' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:03:01.546 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:03:01.547 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:03:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:01 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:01 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:01 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:01 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:01.952 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:01 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:01 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:02.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:01 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:02.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:01 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:02.155 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:01 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:02.155 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:01 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:02.155 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:01 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:02.155 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:01 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:02.629 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 -- 172.21.15.57:0/2813489313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1079c0 msgr2=0x7f4b4c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 --2- 172.21.15.57:0/2813489313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c107da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009860 tx=0x7f4b3c02a340 comp rx=0 tx=0).stop 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 -- 172.21.15.57:0/2813489313 shutdown_connections 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 --2- 172.21.15.57:0/2813489313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 --2- 172.21.15.57:0/2813489313 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 --2- 172.21.15.57:0/2813489313 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c107da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 -- 172.21.15.57:0/2813489313 >> 172.21.15.57:0/2813489313 conn(0x7f4b4c0fd740 msgr2=0x7f4b4c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 -- 172.21.15.57:0/2813489313 shutdown_connections 2024-09-12T03:03:02.630 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.614+0000 7f4b51400640 1 -- 172.21.15.57:0/2813489313 wait complete. 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 Processor -- start 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 -- start start 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111dc0 con 0x7f4b4c1079c0 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111c40 con 0x7f4b4c1019c0 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.615+0000 7f4b51400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111f40 con 0x7f4b4c102360 2024-09-12T03:03:02.631 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b43400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:34506/0 (socket says 172.21.15.57:34506) 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b50a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b43400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:53330/0 (socket says 172.21.15.57:53330) 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 -- 172.21.15.57:0/613309443 learned_addr learned my addr 172.21.15.57:0/613309443 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 -- 172.21.15.57:0/613309443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 msgr2=0x7f4b4c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 -- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1079c0 msgr2=0x7f4b4c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:02.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 -- 172.21.15.57:0/613309443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c19f410 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b43400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.616+0000 7f4b4be00640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c035040 tx=0x7f4b3c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.617+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c037040 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.617+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c19f6a0 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.617+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c040070 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.617+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c003a10 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.617+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1a6fe0 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.618+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c101e20 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.620+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b3c004150 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.621+0000 7f4b49600640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 0x7f4b300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.621+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f4b3c0eebd0 con 0x7f4b4c1019c0 2024-09-12T03:03:02.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.621+0000 7f4b43400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 0x7f4b300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.622+0000 7f4b43400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 0x7f4b300a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b340096c0 tx=0x7f4b34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:02.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.625+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b3c0bbf30 con 0x7f4b4c1019c0 2024-09-12T03:03:02.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:02.828+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b4c1a71b0 con 0x7f4b4c1019c0 2024-09-12T03:03:03.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.403+0000 7f4b49600640 1 -- 172.21.15.57:0/613309443 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f4b3c0bb830 con 0x7f4b4c1019c0 2024-09-12T03:03:03.404 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' created 2024-09-12T03:03:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.407+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 msgr2=0x7f4b300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.407+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 0x7f4b300a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b340096c0 tx=0x7f4b34009260 comp rx=0 tx=0).stop 2024-09-12T03:03:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.407+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 msgr2=0x7f4b4c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:03.408 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.407+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c035040 tx=0x7f4b3c033b60 comp rx=0 tx=0).stop 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 shutdown_connections 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f4b300a2b80 0x7f4b300a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4b4c1079c0 0x7f4b4c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c102360 0x7f4b4c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 --2- 172.21.15.57:0/613309443 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b4c1019c0 0x7f4b4c19a460 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 >> 172.21.15.57:0/613309443 conn(0x7f4b4c0fd740 msgr2=0x7f4b4c10d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 shutdown_connections 2024-09-12T03:03:03.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:03.408+0000 7f4b51400640 1 -- 172.21.15.57:0/613309443 wait complete. 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/613309443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.502 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:03 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/613309443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:03 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/613309443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:03:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:03.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:03 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:04.443 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-12T03:03:04.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:04 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:03:04.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:04 smithi181 ceph-mon[35320]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:03:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:04 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:03:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:04 smithi057 ceph-mon[30361]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:03:04.877 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:04.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:04 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:03:04.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:04 smithi204 ceph-mon[35013]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 -- 172.21.15.57:0/55017838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec103e30 msgr2=0x7f7dec104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 --2- 172.21.15.57:0/55017838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec103e30 0x7f7dec104210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009ca0 tx=0x7f7ddc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 -- 172.21.15.57:0/55017838 shutdown_connections 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 --2- 172.21.15.57:0/55017838 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 0x7f7dec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 --2- 172.21.15.57:0/55017838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec1047e0 0x7f7dec108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 --2- 172.21.15.57:0/55017838 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec103e30 0x7f7dec104210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 -- 172.21.15.57:0/55017838 >> 172.21.15.57:0/55017838 conn(0x7f7dec0ff910 msgr2=0x7f7dec101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 -- 172.21.15.57:0/55017838 shutdown_connections 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.513+0000 7f7debe00640 1 -- 172.21.15.57:0/55017838 wait complete. 2024-09-12T03:03:05.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.514+0000 7f7debe00640 1 Processor -- start 2024-09-12T03:03:05.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.514+0000 7f7debe00640 1 -- start start 2024-09-12T03:03:05.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec103e30 0x7f7dec19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 0x7f7dec19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec114010 con 0x7f7dec1047e0 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec113e90 con 0x7f7dec108f60 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7debe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec114190 con 0x7f7dec103e30 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53666/0 (socket says 172.21.15.57:53666) 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 -- 172.21.15.57:0/1295393522 learned_addr learned my addr 172.21.15.57:0/1295393522 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7deaa00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec103e30 0x7f7dec19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7deb400640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 0x7f7dec19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 -- 172.21.15.57:0/1295393522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec103e30 msgr2=0x7f7dec19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec103e30 0x7f7dec19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 -- 172.21.15.57:0/1295393522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 msgr2=0x7f7dec19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 0x7f7dec19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.515+0000 7f7de2000640 1 -- 172.21.15.57:0/1295393522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19f460 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7de2000640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e9f0 tx=0x7f7dd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400ccc0 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dd400ce60 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19f750 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4010690 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.516+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a7030 con 0x7f7dec1047e0 2024-09-12T03:03:05.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.518+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec105310 con 0x7f7dec1047e0 2024-09-12T03:03:05.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.520+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7dd40046c0 con 0x7f7dec1047e0 2024-09-12T03:03:05.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.521+0000 7f7de3e00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 0x7f7dd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:05.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.521+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f7dd40cf1d0 con 0x7f7dec1047e0 2024-09-12T03:03:05.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.522+0000 7f7deaa00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 0x7f7dd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:05.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.522+0000 7f7deaa00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 0x7f7dd00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:05.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.526+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7dd409c3c0 con 0x7f7dec1047e0 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: pgmap v291: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:05.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:05.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:05.712+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f7dec02d7c0 con 0x7f7dec1047e0 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: pgmap v291: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:05.713 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: pgmap v291: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:05.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:03:06.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.422+0000 7f7de3e00640 1 -- 172.21.15.57:0/1295393522 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f7dd409bcc0 con 0x7f7dec1047e0 2024-09-12T03:03:06.422 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' created 2024-09-12T03:03:06.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 msgr2=0x7f7dd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 0x7f7dd00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc006010 tx=0x7f7ddc009820 comp rx=0 tx=0).stop 2024-09-12T03:03:06.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 msgr2=0x7f7dec19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:06.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e9f0 tx=0x7f7dd400edf0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 shutdown_connections 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7dd00a2b80 0x7f7dd00a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dec108f60 0x7f7dec19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7dec1047e0 0x7f7dec19a9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 --2- 172.21.15.57:0/1295393522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dec103e30 0x7f7dec19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.426+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 >> 172.21.15.57:0/1295393522 conn(0x7f7dec0ff910 msgr2=0x7f7dec100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:06.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.427+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 shutdown_connections 2024-09-12T03:03:06.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:06.427+0000 7f7debe00640 1 -- 172.21.15.57:0/1295393522 wait complete. 2024-09-12T03:03:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:06 smithi181 ceph-mon[35320]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:03:06.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:06 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:03:06.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:06 smithi057 ceph-mon[30361]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:03:06.733 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:06 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:03:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:06 smithi204 ceph-mon[35013]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:03:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:06 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-12T03:03:07.493 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:07 smithi181 ceph-mon[35320]: pgmap v294: 33 pgs: 8 creating+peering, 11 active+clean, 14 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:07 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:03:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:07 smithi181 ceph-mon[35320]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:03:07.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:07 smithi057 ceph-mon[30361]: pgmap v294: 33 pgs: 8 creating+peering, 11 active+clean, 14 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:07 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:03:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:07 smithi057 ceph-mon[30361]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:03:07.870 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:07.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:07 smithi204 ceph-mon[35013]: pgmap v294: 33 pgs: 8 creating+peering, 11 active+clean, 14 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:07.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:07 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1295393522' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-12T03:03:07.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:07 smithi204 ceph-mon[35013]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:03:08.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 -- 172.21.15.57:0/1880370918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0381047e0 msgr2=0x7fb038108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 --2- 172.21.15.57:0/1880370918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0381047e0 0x7fb038108830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb020009900 tx=0x7fb02002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 -- 172.21.15.57:0/1880370918 shutdown_connections 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 --2- 172.21.15.57:0/1880370918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 0x7fb038110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 --2- 172.21.15.57:0/1880370918 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb0381047e0 0x7fb038108830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 --2- 172.21.15.57:0/1880370918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb038103e30 0x7fb038104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 -- 172.21.15.57:0/1880370918 >> 172.21.15.57:0/1880370918 conn(0x7fb0380ff8d0 msgr2=0x7fb038101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 -- 172.21.15.57:0/1880370918 shutdown_connections 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.438+0000 7fb037e00640 1 -- 172.21.15.57:0/1880370918 wait complete. 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.439+0000 7fb037e00640 1 Processor -- start 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 -- start start 2024-09-12T03:03:08.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 0x7fb03819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 0x7fb03819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 0x7fb03819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114010 con 0x7fb038103e30 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb038113e90 con 0x7fb038108f60 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb037e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114190 con 0x7fb0381047e0 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb036a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 0x7fb03819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.440+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 0x7fb03819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb02fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 0x7fb03819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60358/0 (socket says 172.21.15.57:60358) 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb02fe00640 1 -- 172.21.15.57:0/2307996107 learned_addr learned my addr 172.21.15.57:0/2307996107 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb037400640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 0x7fb03819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 -- 172.21.15.57:0/2307996107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 msgr2=0x7fb03819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 0x7fb03819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 -- 172.21.15.57:0/2307996107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 msgr2=0x7fb03819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 0x7fb03819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 -- 172.21.15.57:0/2307996107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819f420 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.441+0000 7fb036a00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 0x7fb03819a470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9f0 tx=0x7fb02800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.443+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02800ccc0 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.443+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb028005020 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.443+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0280049e0 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.443+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819f710 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.443+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a6ff0 con 0x7fb038103e30 2024-09-12T03:03:08.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.444+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb038104210 con 0x7fb038103e30 2024-09-12T03:03:08.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.448+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb028005560 con 0x7fb038103e30 2024-09-12T03:03:08.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.449+0000 7fb02f400640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 0x7fb01c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:08.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.449+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fb0280cee90 con 0x7fb038103e30 2024-09-12T03:03:08.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.449+0000 7fb02fe00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 0x7fb01c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:08.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.450+0000 7fb02fe00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 0x7fb01c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb0200096c0 tx=0x7fb0200046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:08.451 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.452+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0280051d0 con 0x7fb038103e30 2024-09-12T03:03:08.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:08 smithi057 ceph-mon[30361]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:03:08.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:08 smithi057 ceph-mon[30361]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:03:08.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:08.678+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb03819b7d0 con 0x7fb038103e30 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:08 smithi181 ceph-mon[35320]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:03:08.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:08 smithi181 ceph-mon[35320]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:03:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:08 smithi204 ceph-mon[35013]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:03:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:08 smithi204 ceph-mon[35013]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:03:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.450+0000 7fb02f400640 1 -- 172.21.15.57:0/2307996107 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb03819b7d0 con 0x7fb038103e30 2024-09-12T03:03:09.450 INFO:teuthology.orchestra.run.smithi057.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.453+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 msgr2=0x7fb01c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.453+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 0x7fb01c0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb0200096c0 tx=0x7fb0200046c0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.453+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 msgr2=0x7fb03819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.453+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 0x7fb03819a470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9f0 tx=0x7fb02800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 shutdown_connections 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb01c0a2c00 0x7fb01c0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb038108f60 0x7fb03819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0381047e0 0x7fb03819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 --2- 172.21.15.57:0/2307996107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb038103e30 0x7fb03819a470 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 >> 172.21.15.57:0/2307996107 conn(0x7fb0380ff8d0 msgr2=0x7fb038100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 shutdown_connections 2024-09-12T03:03:09.454 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:09.454+0000 7fb037e00640 1 -- 172.21.15.57:0/2307996107 wait complete. 2024-09-12T03:03:09.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:09 smithi181 ceph-mon[35320]: pgmap v297: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:09.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:09 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:03:09.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:09 smithi181 ceph-mon[35320]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:03:09.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:09 smithi057 ceph-mon[30361]: pgmap v297: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:09.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:09 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:03:09.857 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:09 smithi057 ceph-mon[30361]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:03:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:09 smithi204 ceph-mon[35013]: pgmap v297: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:09 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:03:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:09 smithi204 ceph-mon[35013]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T03:03:10.286 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-12T03:03:10.287 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-12T03:03:10.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:10 smithi057 ceph-mon[30361]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:03:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:10 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:03:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:10 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:03:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:10 smithi057 ceph-mon[30361]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:03:10.669 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:10 smithi181 ceph-mon[35320]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:03:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:10 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:03:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:10 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:03:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:10 smithi181 ceph-mon[35320]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:03:10.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:10 smithi204 ceph-mon[35013]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:03:10.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:10 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2307996107' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:03:10.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:10 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T03:03:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:10 smithi204 ceph-mon[35013]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 -- 172.21.15.57:0/4007301424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2010e3c0 msgr2=0x7f9a20110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 --2- 172.21.15.57:0/4007301424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2010e3c0 0x7f9a20110810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009be0 tx=0x7f9a1802a750 comp rx=0 tx=0).stop 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 -- 172.21.15.57:0/4007301424 shutdown_connections 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 --2- 172.21.15.57:0/4007301424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a2010e3c0 0x7f9a20110810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 --2- 172.21.15.57:0/4007301424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074f40 0x7f9a2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 --2- 172.21.15.57:0/4007301424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a20074620 0x7f9a20074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 -- 172.21.15.57:0/4007301424 >> 172.21.15.57:0/4007301424 conn(0x7f9a200fd8c0 msgr2=0x7f9a200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:11.277 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.277+0000 7f9a26600640 1 -- 172.21.15.57:0/4007301424 shutdown_connections 2024-09-12T03:03:11.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.278+0000 7f9a26600640 1 -- 172.21.15.57:0/4007301424 wait complete. 2024-09-12T03:03:11.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.279+0000 7f9a26600640 1 Processor -- start 2024-09-12T03:03:11.278 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.279+0000 7f9a26600640 1 -- start start 2024-09-12T03:03:11.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.279+0000 7f9a26600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074620 0x7f9a2019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:11.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.279+0000 7f9a26600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a20074f40 0x7f9a2019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:11.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a26600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:11.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a26600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20114170 con 0x7f9a2010e3c0 2024-09-12T03:03:11.279 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a26600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9a20113ff0 con 0x7f9a20074f40 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a26600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a201142f0 con 0x7f9a20074620 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074620 0x7f9a2019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39830/0 (socket says 172.21.15.57:39830) 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 -- 172.21.15.57:0/1270696507 learned_addr learned my addr 172.21.15.57:0/1270696507 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:11.280 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1ea00640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a20074f40 0x7f9a2019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 -- 172.21.15.57:0/1270696507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074620 msgr2=0x7f9a2019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074620 0x7f9a2019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 -- 172.21.15.57:0/1270696507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a20074f40 msgr2=0x7f9a2019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a20074f40 0x7f9a2019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.280+0000 7f9a1fe00640 1 -- 172.21.15.57:0/1270696507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2019f600 con 0x7f9a2010e3c0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a1fe00640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009b30 tx=0x7f9a180305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18041020 con 0x7f9a2010e3c0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a18003e50 con 0x7f9a2010e3c0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2019f890 con 0x7f9a2010e3c0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a18036040 con 0x7f9a2010e3c0 2024-09-12T03:03:11.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.281+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a71d0 con 0x7f9a2010e3c0 2024-09-12T03:03:11.282 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.282+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a20075cd0 con 0x7f9a2010e3c0 2024-09-12T03:03:11.283 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.284+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9a18044020 con 0x7f9a2010e3c0 2024-09-12T03:03:11.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.285+0000 7f9a17e00640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:11.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.285+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9a180efbf0 con 0x7f9a2010e3c0 2024-09-12T03:03:11.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.285+0000 7f9a1f400640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:11.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.286+0000 7f9a1f400640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 0x7f9a040a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009530 tx=0x7f9a10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:11.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.289+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a1804b020 con 0x7f9a2010e3c0 2024-09-12T03:03:11.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:11.485+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f9a20074a00 con 0x7f9a2010e3c0 2024-09-12T03:03:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: pgmap v300: 97 pgs: 13 creating+peering, 62 active+clean, 22 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-12T03:03:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:03:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: Cluster is now healthy 2024-09-12T03:03:11.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:active 2024-09-12T03:03:11.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:11.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:11 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: pgmap v300: 97 pgs: 13 creating+peering, 62 active+clean, 22 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-12T03:03:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:03:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: Cluster is now healthy 2024-09-12T03:03:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:active 2024-09-12T03:03:11.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:11.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:11 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: pgmap v300: 97 pgs: 13 creating+peering, 62 active+clean, 22 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: Cluster is now healthy 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] up:active 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:11.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:11 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:12.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.463+0000 7f9a17e00640 1 -- 172.21.15.57:0/1270696507 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f9a20074a00 con 0x7f9a2010e3c0 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 msgr2=0x7f9a040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 0x7f9a040a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009530 tx=0x7f9a10009260 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 msgr2=0x7f9a2019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009b30 tx=0x7f9a180305d0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 shutdown_connections 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9a040a2b80 0x7f9a040a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9a2010e3c0 0x7f9a2019ef20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a20074f40 0x7f9a2019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 --2- 172.21.15.57:0/1270696507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a20074620 0x7f9a2019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:12.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 >> 172.21.15.57:0/1270696507 conn(0x7f9a200fd8c0 msgr2=0x7f9a200fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:12.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 shutdown_connections 2024-09-12T03:03:12.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:12.467+0000 7f9a26600640 1 -- 172.21.15.57:0/1270696507 wait complete. 2024-09-12T03:03:12.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:12 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:03:12.703 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:03:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: 2024-09-12T03:03:12.409+0000 7f61664d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:03:12.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:12 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:03:12.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:12 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-12T03:03:13.379 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:03:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:13 smithi057 ceph-mon[30361]: pgmap v301: 97 pgs: 2 creating+peering, 95 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-12T03:03:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:13 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:03:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:13 smithi057 ceph-mon[30361]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:03:13.761 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:13.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:13 smithi204 ceph-mon[35013]: pgmap v301: 97 pgs: 2 creating+peering, 95 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-12T03:03:13.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:13 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:03:13.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:13 smithi204 ceph-mon[35013]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:03:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:13 smithi181 ceph-mon[35320]: pgmap v301: 97 pgs: 2 creating+peering, 95 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 6 op/s 2024-09-12T03:03:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:13 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1270696507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-12T03:03:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:13 smithi181 ceph-mon[35320]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 -- 172.21.15.57:0/4014927543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c1027a0 msgr2=0x7fe65c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 --2- 172.21.15.57:0/4014927543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c1027a0 0x7fe65c1067f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe644009900 tx=0x7fe64402a410 comp rx=0 tx=0).stop 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 -- 172.21.15.57:0/4014927543 shutdown_connections 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 --2- 172.21.15.57:0/4014927543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c106f20 0x7fe65c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 --2- 172.21.15.57:0/4014927543 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c1027a0 0x7fe65c1067f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:14.392 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 --2- 172.21.15.57:0/4014927543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:14.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 -- 172.21.15.57:0/4014927543 >> 172.21.15.57:0/4014927543 conn(0x7fe65c0fd8f0 msgr2=0x7fe65c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:14.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 -- 172.21.15.57:0/4014927543 shutdown_connections 2024-09-12T03:03:14.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.390+0000 7fe661a00640 1 -- 172.21.15.57:0/4014927543 wait complete. 2024-09-12T03:03:14.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.391+0000 7fe661a00640 1 Processor -- start 2024-09-12T03:03:14.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.391+0000 7fe661a00640 1 -- start start 2024-09-12T03:03:14.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:14.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 0x7fe65c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:14.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:14.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111f80 con 0x7fe65c106f20 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111e00 con 0x7fe65c101df0 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe661a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c112100 con 0x7fe65c1027a0 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe65b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe65b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39842/0 (socket says 172.21.15.57:39842) 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe65b400640 1 -- 172.21.15.57:0/2364232180 learned_addr learned my addr 172.21.15.57:0/2364232180 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe652000640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 0x7fe65c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.392+0000 7fe65aa00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 -- 172.21.15.57:0/2364232180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 msgr2=0x7fe65c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 0x7fe65c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 -- 172.21.15.57:0/2364232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 msgr2=0x7fe65c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 -- 172.21.15.57:0/2364232180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c1a6960 con 0x7fe65c106f20 2024-09-12T03:03:14.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe652000640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 0x7fe65c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65aa00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe65b400640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe65400d750 tx=0x7fe65400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654017070 con 0x7fe65c106f20 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe654005500 con 0x7fe65c106f20 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c1a6c50 con 0x7fe65c106f20 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.393+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654004d90 con 0x7fe65c106f20 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.394+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a7160 con 0x7fe65c106f20 2024-09-12T03:03:14.396 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.395+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c1021d0 con 0x7fe65c106f20 2024-09-12T03:03:14.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.397+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe654004f30 con 0x7fe65c106f20 2024-09-12T03:03:14.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.398+0000 7fe653e00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 0x7fe6400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:14.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.398+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fe6540ce910 con 0x7fe65c106f20 2024-09-12T03:03:14.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.398+0000 7fe65aa00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 0x7fe6400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:14.400 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.399+0000 7fe65aa00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 0x7fe6400a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe64c005fd0 tx=0x7fe64c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.407+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe65409b190 con 0x7fe65c106f20 2024-09-12T03:03:14.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:14.596+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe65c10a160 con 0x7fe65c106f20 2024-09-12T03:03:15.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:15 smithi057 ceph-mon[30361]: pgmap v303: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:03:15.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:15 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:03:15.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:15 smithi204 ceph-mon[35013]: pgmap v303: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:03:15.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:15 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:03:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:15 smithi181 ceph-mon[35320]: pgmap v303: 97 pgs: 97 active+clean; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-12T03:03:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:15 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:03:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.494+0000 7fe653e00640 1 -- 172.21.15.57:0/2364232180 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fe65409f490 con 0x7fe65c106f20 2024-09-12T03:03:16.496 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' created 2024-09-12T03:03:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 msgr2=0x7fe6400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 0x7fe6400a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe64c005fd0 tx=0x7fe64c009500 comp rx=0 tx=0).stop 2024-09-12T03:03:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 msgr2=0x7fe65c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe65400d750 tx=0x7fe65400dc10 comp rx=0 tx=0).stop 2024-09-12T03:03:16.499 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 shutdown_connections 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6400a2c00 0x7fe6400a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe65c106f20 0x7fe65c074f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe65c1027a0 0x7fe65c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 --2- 172.21.15.57:0/2364232180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe65c101df0 0x7fe65c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 >> 172.21.15.57:0/2364232180 conn(0x7fe65c0fd8f0 msgr2=0x7fe65c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.498+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 shutdown_connections 2024-09-12T03:03:16.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:16.499+0000 7fe661a00640 1 -- 172.21.15.57:0/2364232180 wait complete. 2024-09-12T03:03:16.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:16 smithi057 ceph-mon[30361]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:03:16.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:16 smithi204 ceph-mon[35013]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:03:16.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:16 smithi181 ceph-mon[35320]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T03:03:17.419 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-12T03:03:17.821 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:17.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:17 smithi057 ceph-mon[30361]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-12T03:03:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:17 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:03:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:17 smithi057 ceph-mon[30361]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:03:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:17 smithi204 ceph-mon[35013]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-12T03:03:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:17 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:03:17.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:17 smithi204 ceph-mon[35013]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:03:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:17 smithi181 ceph-mon[35320]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-12T03:03:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:17 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2364232180' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:03:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:17 smithi181 ceph-mon[35320]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T03:03:18.455 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 -- 172.21.15.57:0/240674397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c102360 msgr2=0x7f935c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 --2- 172.21.15.57:0/240674397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c102360 0x7f935c10e890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9358009cd0 tx=0x7f935802a8f0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 -- 172.21.15.57:0/240674397 shutdown_connections 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 --2- 172.21.15.57:0/240674397 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c102360 0x7f935c10e890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 --2- 172.21.15.57:0/240674397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1019c0 0x7f935c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 --2- 172.21.15.57:0/240674397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c1079c0 0x7f935c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 -- 172.21.15.57:0/240674397 >> 172.21.15.57:0/240674397 conn(0x7f935c0fd6e0 msgr2=0x7f935c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 -- 172.21.15.57:0/240674397 shutdown_connections 2024-09-12T03:03:18.456 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.454+0000 7f9363200640 1 -- 172.21.15.57:0/240674397 wait complete. 2024-09-12T03:03:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.455+0000 7f9363200640 1 Processor -- start 2024-09-12T03:03:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 -- start start 2024-09-12T03:03:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 0x7f935c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1079c0 0x7f935c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f935c111dc0 con 0x7f935c1019c0 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f935c111c40 con 0x7f935c102360 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9363200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f935c111f40 con 0x7f935c1079c0 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.456+0000 7f9361e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 0x7f935c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33986/0 (socket says 172.21.15.57:33986) 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 -- 172.21.15.57:0/3722076892 learned_addr learned my addr 172.21.15.57:0/3722076892 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9362800640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1079c0 0x7f935c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 -- 172.21.15.57:0/3722076892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1079c0 msgr2=0x7f935c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1079c0 0x7f935c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 -- 172.21.15.57:0/3722076892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 msgr2=0x7f935c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 0x7f935c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 -- 172.21.15.57:0/3722076892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c19d1b0 con 0x7f935c1019c0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361400640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 0x7f935c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.457+0000 7f9361e00640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9f0 tx=0x7f935000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.458+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935000cd80 con 0x7f935c1019c0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.458+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9350010910 con 0x7f935c1019c0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.458+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c19d4a0 con 0x7f935c1019c0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.458+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350018990 con 0x7f935c1019c0 2024-09-12T03:03:18.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.458+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c1a4d80 con 0x7f935c1019c0 2024-09-12T03:03:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.459+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c101e20 con 0x7f935c1019c0 2024-09-12T03:03:18.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.461+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9350010ac0 con 0x7f935c1019c0 2024-09-12T03:03:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.462+0000 7f9356a00640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 0x7f93440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.462+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f93500cf490 con 0x7f935c1019c0 2024-09-12T03:03:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.462+0000 7f9361400640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 0x7f93440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.463+0000 7f9361400640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 0x7f93440a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f935c106a10 tx=0x7f934c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:18.467 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.467+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f935009bb20 con 0x7f935c1019c0 2024-09-12T03:03:18.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:18 smithi057 ceph-mon[30361]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:03:18.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:18.659+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f935c199560 con 0x7f935c1019c0 2024-09-12T03:03:18.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:18 smithi204 ceph-mon[35013]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:03:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:18 smithi181 ceph-mon[35320]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T03:03:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.530+0000 7f9356a00640 1 -- 172.21.15.57:0/3722076892 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) ==== 169+0+0 (secure 0 0 0) 0x7f935009fe20 con 0x7f935c1019c0 2024-09-12T03:03:19.531 INFO:teuthology.orchestra.run.smithi057.stderr:set pool 4 allow_ec_overwrites to true 2024-09-12T03:03:19.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.533+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 msgr2=0x7f93440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.533+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 0x7f93440a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f935c106a10 tx=0x7f934c005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 msgr2=0x7f935c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9f0 tx=0x7f935000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 shutdown_connections 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93440a2c00 0x7f93440a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f935c1079c0 0x7f935c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f935c102360 0x7f935c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 --2- 172.21.15.57:0/3722076892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f935c1019c0 0x7f935c198200 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:19.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.534+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 >> 172.21.15.57:0/3722076892 conn(0x7f935c0fd6e0 msgr2=0x7f935c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.535+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 shutdown_connections 2024-09-12T03:03:19.536 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:19.535+0000 7f9363200640 1 -- 172.21.15.57:0/3722076892 wait complete. 2024-09-12T03:03:19.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:19 smithi204 ceph-mon[35013]: pgmap v308: 129 pgs: 10 creating+peering, 19 unknown, 100 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-09-12T03:03:19.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:19 smithi204 ceph-mon[35013]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:19.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:19 smithi204 ceph-mon[35013]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:03:19.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:19 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:03:19.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:19 smithi057 ceph-mon[30361]: pgmap v308: 129 pgs: 10 creating+peering, 19 unknown, 100 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-09-12T03:03:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:19 smithi057 ceph-mon[30361]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:19 smithi057 ceph-mon[30361]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:03:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:19 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:03:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:19 smithi181 ceph-mon[35320]: pgmap v308: 129 pgs: 10 creating+peering, 19 unknown, 100 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 195 B/s wr, 0 op/s 2024-09-12T03:03:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:19 smithi181 ceph-mon[35320]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:03:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:19 smithi181 ceph-mon[35320]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T03:03:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:19 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-12T03:03:20.439 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-12T03:03:20.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:20 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:03:20.811 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:20 smithi204 ceph-mon[35013]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:03:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:20.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:20 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:03:20.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:20 smithi057 ceph-mon[30361]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:03:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:20 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3722076892' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-12T03:03:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:20 smithi181 ceph-mon[35320]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T03:03:21.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 -- 172.21.15.57:0/1794389675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8109120 msgr2=0x7f48b8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 --2- 172.21.15.57:0/1794389675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8109120 0x7f48b8110ca0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009be0 tx=0x7f48b002a6c0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 -- 172.21.15.57:0/1794389675 shutdown_connections 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 --2- 172.21.15.57:0/1794389675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8109120 0x7f48b8110ca0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 --2- 172.21.15.57:0/1794389675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 0x7f48b81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 --2- 172.21.15.57:0/1794389675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8103ff0 0x7f48b81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.425+0000 7f48c0600640 1 -- 172.21.15.57:0/1794389675 >> 172.21.15.57:0/1794389675 conn(0x7f48b80ffa90 msgr2=0x7f48b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.426+0000 7f48c0600640 1 -- 172.21.15.57:0/1794389675 shutdown_connections 2024-09-12T03:03:21.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.426+0000 7f48c0600640 1 -- 172.21.15.57:0/1794389675 wait complete. 2024-09-12T03:03:21.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.426+0000 7f48c0600640 1 Processor -- start 2024-09-12T03:03:21.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 -- start start 2024-09-12T03:03:21.430 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 0x7f48b819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8109120 0x7f48b819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f48b8114130 con 0x7f48b81049a0 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f48b8113fb0 con 0x7f48b8109120 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48c0600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f48b81142b0 con 0x7f48b8103ff0 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48bdc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48bdc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51164/0 (socket says 172.21.15.57:51164) 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48bdc00640 1 -- 172.21.15.57:0/499458972 learned_addr learned my addr 172.21.15.57:0/499458972 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:21.431 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48bd200640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 0x7f48b819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.427+0000 7f48be600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8109120 0x7f48b819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 -- 172.21.15.57:0/499458972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8109120 msgr2=0x7f48b819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8109120 0x7f48b819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 -- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 msgr2=0x7f48b819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 0x7f48b819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 -- 172.21.15.57:0/499458972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b819f540 con 0x7f48b8103ff0 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48bdc00640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00d950 tx=0x7f48ac00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac017030 con 0x7f48b8103ff0 2024-09-12T03:03:21.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ac004020 con 0x7f48b8103ff0 2024-09-12T03:03:21.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac005650 con 0x7f48b8103ff0 2024-09-12T03:03:21.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.428+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b819f830 con 0x7f48b8103ff0 2024-09-12T03:03:21.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.429+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b81a7110 con 0x7f48b8103ff0 2024-09-12T03:03:21.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.430+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b81043d0 con 0x7f48b8103ff0 2024-09-12T03:03:21.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.431+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48ac0041d0 con 0x7f48b8103ff0 2024-09-12T03:03:21.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.433+0000 7f48b6000640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 0x7f48a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:21.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.433+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7f48ac0cee30 con 0x7f48b8103ff0 2024-09-12T03:03:21.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.433+0000 7f48bd200640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 0x7f48a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:21.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.434+0000 7f48bd200640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 0x7f48a00a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48b80ffeb0 tx=0x7f48a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:21.435 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.435+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48ac09bb90 con 0x7f48b8103ff0 2024-09-12T03:03:21.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:21 smithi057 ceph-mon[30361]: pgmap v311: 129 pgs: 10 creating+peering, 11 unknown, 108 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:21.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:21.659+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f48b81a7440 con 0x7f48b8103ff0 2024-09-12T03:03:21.854 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:21 smithi181 ceph-mon[35320]: pgmap v311: 129 pgs: 10 creating+peering, 11 unknown, 108 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:21 smithi204 ceph-mon[35013]: pgmap v311: 129 pgs: 10 creating+peering, 11 unknown, 108 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.560+0000 7f48b6000640 1 -- 172.21.15.57:0/499458972 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f48ac09b490 con 0x7f48b8103ff0 2024-09-12T03:03:22.561 INFO:teuthology.orchestra.run.smithi057.stderr:added data pool 4 to fsmap 2024-09-12T03:03:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.563+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 msgr2=0x7f48a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.563+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 0x7f48a00a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48b80ffeb0 tx=0x7f48a4009260 comp rx=0 tx=0).stop 2024-09-12T03:03:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 msgr2=0x7f48b819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:22.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00d950 tx=0x7f48ac00dd50 comp rx=0 tx=0).stop 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 shutdown_connections 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f48a00a2c00 0x7f48a00a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48b8109120 0x7f48b819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f48b81049a0 0x7f48b819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 --2- 172.21.15.57:0/499458972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48b8103ff0 0x7f48b819a590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 >> 172.21.15.57:0/499458972 conn(0x7f48b80ffa90 msgr2=0x7f48b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 shutdown_connections 2024-09-12T03:03:22.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:22.564+0000 7f48c0600640 1 -- 172.21.15.57:0/499458972 wait complete. 2024-09-12T03:03:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:22 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/499458972' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:22.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:22 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:22.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:22 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/499458972' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:22.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:22 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:22 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/499458972' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:22 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-12T03:03:23.480 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:23.861 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:23 smithi057 ceph-mon[30361]: pgmap v312: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:23 smithi057 ceph-mon[30361]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:03:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:23 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:03:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:23 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:23 smithi204 ceph-mon[35013]: pgmap v312: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:23 smithi204 ceph-mon[35013]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:03:23.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:23 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:03:23.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:23 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:23 smithi181 ceph-mon[35320]: pgmap v312: 129 pgs: 7 creating+peering, 122 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:23 smithi181 ceph-mon[35320]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T03:03:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:23 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-12T03:03:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:23 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:24.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 -- 172.21.15.57:0/3564018503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 msgr2=0x7f151c112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:24.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 --2- 172.21.15.57:0/3564018503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c112510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f150c009860 tx=0x7f150c02a340 comp rx=0 tx=0).stop 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 -- 172.21.15.57:0/3564018503 shutdown_connections 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 --2- 172.21.15.57:0/3564018503 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 0x7f151c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 --2- 172.21.15.57:0/3564018503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c112510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 --2- 172.21.15.57:0/3564018503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 -- 172.21.15.57:0/3564018503 >> 172.21.15.57:0/3564018503 conn(0x7f151c0ffa50 msgr2=0x7f151c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 -- 172.21.15.57:0/3564018503 shutdown_connections 2024-09-12T03:03:24.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.440+0000 7f1524800640 1 -- 172.21.15.57:0/3564018503 wait complete. 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.441+0000 7f1524800640 1 Processor -- start 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 -- start start 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 0x7f151c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1185c0 con 0x7f151c112a50 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f151c118440 con 0x7f151c076720 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1524800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f151c118740 con 0x7f151c077040 2024-09-12T03:03:24.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1522800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 0x7f151c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1521e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1521e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:59108/0 (socket says 172.21.15.57:59108) 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1521e00640 1 -- 172.21.15.57:0/3773481717 learned_addr learned my addr 172.21.15.57:0/3773481717 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.442+0000 7f1521400640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 -- 172.21.15.57:0/3773481717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 msgr2=0x7f151c1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c1a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 -- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 msgr2=0x7f151c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 0x7f151c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 -- 172.21.15.57:0/3773481717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c19f020 con 0x7f151c076720 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1521e00640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f151000e9f0 tx=0x7f151000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151000cd80 con 0x7f151c076720 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510010910 con 0x7f151c076720 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15100188f0 con 0x7f151c076720 2024-09-12T03:03:24.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.443+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c19f310 con 0x7f151c076720 2024-09-12T03:03:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.444+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1ab600 con 0x7f151c076720 2024-09-12T03:03:24.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.444+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c076b00 con 0x7f151c076720 2024-09-12T03:03:24.447 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.447+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1510010ac0 con 0x7f151c076720 2024-09-12T03:03:24.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.448+0000 7f1516a00640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 0x7f15040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:24.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.448+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f15100cfa70 con 0x7f151c076720 2024-09-12T03:03:24.448 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.448+0000 7f1521400640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 0x7f15040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:24.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.449+0000 7f1521400640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 0x7f15040a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f150c02a930 tx=0x7f150c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:24.450 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.450+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f151009c0e0 con 0x7f151c076720 2024-09-12T03:03:24.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.673+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f151c1ab830 con 0x7f151c076720 2024-09-12T03:03:24.675 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.674+0000 7f1516a00640 1 -- 172.21.15.57:0/3773481717 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7f15100a03e0 con 0x7f151c076720 2024-09-12T03:03:24.676 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:24.676 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"btime":"2024-09-12T03:03:22:546095+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:21.662051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-09-12T03:03:24.676 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 msgr2=0x7f15040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 0x7f15040a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f150c02a930 tx=0x7f150c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 msgr2=0x7f151c1a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f151000e9f0 tx=0x7f151000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 shutdown_connections 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f15040a2b80 0x7f15040a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f151c112a50 0x7f151c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c077040 0x7f151c1a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 --2- 172.21.15.57:0/3773481717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f151c076720 0x7f151c1a4730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 >> 172.21.15.57:0/3773481717 conn(0x7f151c0ffa50 msgr2=0x7f151c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:24.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 shutdown_connections 2024-09-12T03:03:24.679 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:24.677+0000 7f1524800640 1 -- 172.21.15.57:0/3773481717 wait complete. 2024-09-12T03:03:24.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:24 smithi057 ceph-mon[30361]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:03:24.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:24 smithi057 ceph-mon[30361]: Cluster is now healthy 2024-09-12T03:03:24.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:24 smithi204 ceph-mon[35013]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:03:24.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:24 smithi204 ceph-mon[35013]: Cluster is now healthy 2024-09-12T03:03:24.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:24 smithi181 ceph-mon[35320]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T03:03:24.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:24 smithi181 ceph-mon[35320]: Cluster is now healthy 2024-09-12T03:03:25.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:25.745 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:25 smithi057 ceph-mon[30361]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:25.745 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:25 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3773481717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:25 smithi204 ceph-mon[35013]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:25.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:25 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3773481717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:25.896 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:25 smithi181 ceph-mon[35320]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:25 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3773481717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 -- 172.21.15.57:0/436284793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd810e3c0 msgr2=0x7f1cd8110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 --2- 172.21.15.57:0/436284793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd810e3c0 0x7f1cd8110810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009ca0 tx=0x7f1cd002a950 comp rx=0 tx=0).stop 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 -- 172.21.15.57:0/436284793 shutdown_connections 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 --2- 172.21.15.57:0/436284793 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd810e3c0 0x7f1cd8110810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 --2- 172.21.15.57:0/436284793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd8074f40 0x7f1cd810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 --2- 172.21.15.57:0/436284793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 -- 172.21.15.57:0/436284793 >> 172.21.15.57:0/436284793 conn(0x7f1cd80fd8c0 msgr2=0x7f1cd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:26.515 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 -- 172.21.15.57:0/436284793 shutdown_connections 2024-09-12T03:03:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.513+0000 7f1cde200640 1 -- 172.21.15.57:0/436284793 wait complete. 2024-09-12T03:03:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.514+0000 7f1cde200640 1 Processor -- start 2024-09-12T03:03:26.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.514+0000 7f1cde200640 1 -- start start 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 0x7f1cd819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 0x7f1cd819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd81141a0 con 0x7f1cd8074f40 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8114020 con 0x7f1cd810e3c0 2024-09-12T03:03:26.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cde200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8114320 con 0x7f1cd8074620 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1ccfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 0x7f1cd819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cd7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cd7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 0x7f1cd819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1ccfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 0x7f1cd819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34038/0 (socket says 172.21.15.57:34038) 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1cd7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51208/0 (socket says 172.21.15.57:51208) 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.515+0000 7f1ccfe00640 1 -- 172.21.15.57:0/583900459 learned_addr learned my addr 172.21.15.57:0/583900459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 -- 172.21.15.57:0/583900459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 msgr2=0x7f1cd819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 -- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 msgr2=0x7f1cd819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 0x7f1cd819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 -- 172.21.15.57:0/583900459 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd819f5f0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd7e00640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 0x7f1cd819ef10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0005e30 tx=0x7f1cd0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:26.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0034040 con 0x7f1cd810e3c0 2024-09-12T03:03:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd002f940 con 0x7f1cd810e3c0 2024-09-12T03:03:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd819f880 con 0x7f1cd810e3c0 2024-09-12T03:03:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.516+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00045e0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.517+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81a71c0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.518+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8075cd0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.520 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.520+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1cd0037020 con 0x7f1cd810e3c0 2024-09-12T03:03:26.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.521+0000 7f1cd4c00640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 0x7f1cbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:26.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.521+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f1cd00eff20 con 0x7f1cd810e3c0 2024-09-12T03:03:26.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.521+0000 7f1cd7400640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 0x7f1cbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:26.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.522+0000 7f1cd7400640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 0x7f1cbc0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009bd0 tx=0x7f1cc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:26.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.523+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cd00bcbe0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.712 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.711+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cd8109540 con 0x7f1cd810e3c0 2024-09-12T03:03:26.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.713+0000 7f1cd4c00640 1 -- 172.21.15.57:0/583900459 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24199 (secure 0 0 0) 0x7f1cd00bc4e0 con 0x7f1cd810e3c0 2024-09-12T03:03:26.714 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:26.714 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:21.661916+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.715+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 msgr2=0x7f1cbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.715+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 0x7f1cbc0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009bd0 tx=0x7f1cc8005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:26.716 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.715+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 msgr2=0x7f1cd819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.715+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 0x7f1cd819ef10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0005e30 tx=0x7f1cd0003b80 comp rx=0 tx=0).stop 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 shutdown_connections 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1cbc0a2b80 0x7f1cbc0a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cd810e3c0 0x7f1cd819ef10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cd8074f40 0x7f1cd819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 --2- 172.21.15.57:0/583900459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cd8074620 0x7f1cd819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 >> 172.21.15.57:0/583900459 conn(0x7f1cd80fd8c0 msgr2=0x7f1cd80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 shutdown_connections 2024-09-12T03:03:26.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:26.716+0000 7f1cde200640 1 -- 172.21.15.57:0/583900459 wait complete. 2024-09-12T03:03:27.637 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:27.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:27 smithi057 ceph-mon[30361]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:27.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:27 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/583900459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:27 smithi204 ceph-mon[35013]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:27 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/583900459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:27 smithi181 ceph-mon[35320]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:27 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/583900459' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:28.016 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:28.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.660+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2635492550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b0104010 msgr2=0x7fe5b01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.660+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2635492550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b0104010 0x7fe5b01043f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009900 tx=0x7fe5a402a410 comp rx=0 tx=0).stop 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2635492550 shutdown_connections 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2635492550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0109140 0x7fe5b0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2635492550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b01049c0 0x7fe5b0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2635492550 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b0104010 0x7fe5b01043f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2635492550 >> 172.21.15.57:0/2635492550 conn(0x7fe5b00ffa50 msgr2=0x7fe5b0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2635492550 shutdown_connections 2024-09-12T03:03:28.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.661+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2635492550 wait complete. 2024-09-12T03:03:28.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 Processor -- start 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 -- start start 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0104010 0x7fe5b019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b0109140 0x7fe5b019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0114190 con 0x7fe5b01049c0 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0114010 con 0x7fe5b0104010 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0114310 con 0x7fe5b0109140 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.662+0000 7fe5b4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39154/0 (socket says 172.21.15.57:39154) 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 -- 172.21.15.57:0/2830589295 learned_addr learned my addr 172.21.15.57:0/2830589295 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:28.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b5400640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0104010 0x7fe5b019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b5e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b0109140 0x7fe5b019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 -- 172.21.15.57:0/2830589295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b0109140 msgr2=0x7fe5b019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b0109140 0x7fe5b019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 -- 172.21.15.57:0/2830589295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0104010 msgr2=0x7fe5b019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0104010 0x7fe5b019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 -- 172.21.15.57:0/2830589295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b019f5b0 con 0x7fe5b01049c0 2024-09-12T03:03:28.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5b4a00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:28.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.663+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00cd80 con 0x7fe5b01049c0 2024-09-12T03:03:28.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.664+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c010910 con 0x7fe5b01049c0 2024-09-12T03:03:28.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.664+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b019f8a0 con 0x7fe5b01049c0 2024-09-12T03:03:28.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.664+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c018990 con 0x7fe5b01049c0 2024-09-12T03:03:28.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.664+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a7180 con 0x7fe5b01049c0 2024-09-12T03:03:28.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.665+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b01043f0 con 0x7fe5b01049c0 2024-09-12T03:03:28.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.667+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe59c010ac0 con 0x7fe5b01049c0 2024-09-12T03:03:28.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.669+0000 7fe5ae000640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 0x7fe5980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:28.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.669+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe59c0cf550 con 0x7fe5b01049c0 2024-09-12T03:03:28.669 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.669+0000 7fe5b5400640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 0x7fe5980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:28.670 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.670+0000 7fe5b5400640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 0x7fe5980a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:28.674 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.673+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe59c09bbc0 con 0x7fe5b01049c0 2024-09-12T03:03:28.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.858+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5b002d7c0 con 0x7fe5b01049c0 2024-09-12T03:03:28.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.859+0000 7fe5ae000640 1 -- 172.21.15.57:0/2830589295 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24199 (secure 0 0 0) 0x7fe59c09fec0 con 0x7fe5b01049c0 2024-09-12T03:03:28.860 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:28.861 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:21.661916+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 msgr2=0x7fe5980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 0x7fe5980a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a4005b50 comp rx=0 tx=0).stop 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 msgr2=0x7fe5b019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00e9f0 tx=0x7fe59c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 shutdown_connections 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe5980a2c00 0x7fe5980a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5b0109140 0x7fe5b019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe5b01049c0 0x7fe5b019ab40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 --2- 172.21.15.57:0/2830589295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b0104010 0x7fe5b019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 >> 172.21.15.57:0/2830589295 conn(0x7fe5b00ffa50 msgr2=0x7fe5b0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 shutdown_connections 2024-09-12T03:03:28.864 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:28.862+0000 7fe5b7e00640 1 -- 172.21.15.57:0/2830589295 wait complete. 2024-09-12T03:03:29.696 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:29 smithi057 ceph-mon[30361]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:29 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2830589295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:29 smithi204 ceph-mon[35013]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:29 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2830589295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:29 smithi181 ceph-mon[35320]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:29 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2830589295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:30.076 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 -- 172.21.15.57:0/1751733088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 msgr2=0x7ff228110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 --2- 172.21.15.57:0/1751733088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff228110cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff220009ca0 tx=0x7ff22002a8b0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 -- 172.21.15.57:0/1751733088 shutdown_connections 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 --2- 172.21.15.57:0/1751733088 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff228110cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 --2- 172.21.15.57:0/1751733088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2281049c0 0x7ff228108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 --2- 172.21.15.57:0/1751733088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff228104010 0x7ff2281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 -- 172.21.15.57:0/1751733088 >> 172.21.15.57:0/1751733088 conn(0x7ff2280ffa50 msgr2=0x7ff228101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:30.681 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 -- 172.21.15.57:0/1751733088 shutdown_connections 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.676+0000 7ff22e600640 1 -- 172.21.15.57:0/1751733088 wait complete. 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.677+0000 7ff22e600640 1 Processor -- start 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.677+0000 7ff22e600640 1 -- start start 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff228104010 0x7ff22819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 0x7ff22819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff2281141c0 con 0x7ff228109140 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff228114040 con 0x7ff228104010 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff22e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff228114340 con 0x7ff2281049c0 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff227e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff227e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39180/0 (socket says 172.21.15.57:39180) 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff227e00640 1 -- 172.21.15.57:0/1770189164 learned_addr learned my addr 172.21.15.57:0/1770189164 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:30.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff227400640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff228104010 0x7ff22819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 0x7ff22819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 -- 172.21.15.57:0/1770189164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff228104010 msgr2=0x7ff22819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff228104010 0x7ff22819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 -- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 msgr2=0x7ff22819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.678+0000 7ff226a00640 1 -- 172.21.15.57:0/1770189164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22819f670 con 0x7ff2281049c0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff227e00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff226a00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 0x7ff22819ac00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e9f0 tx=0x7ff21400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400ccc0 con 0x7ff2281049c0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff214010910 con 0x7ff2281049c0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22819f960 con 0x7ff2281049c0 2024-09-12T03:03:30.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214018960 con 0x7ff2281049c0 2024-09-12T03:03:30.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.679+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2281a7240 con 0x7ff2281049c0 2024-09-12T03:03:30.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.680+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2281043f0 con 0x7ff2281049c0 2024-09-12T03:03:30.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.682+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff214010ac0 con 0x7ff2281049c0 2024-09-12T03:03:30.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.684+0000 7ff21fe00640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 0x7ff20c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:30.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.684+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7ff2140cf9b0 con 0x7ff2281049c0 2024-09-12T03:03:30.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.684+0000 7ff227400640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 0x7ff20c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:30.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.684+0000 7ff227400640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 0x7ff20c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff218009bd0 tx=0x7ff218005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:30.688 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.686+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff21409c020 con 0x7ff2281049c0 2024-09-12T03:03:30.908 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.908+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff22819ba20 con 0x7ff2281049c0 2024-09-12T03:03:30.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.909+0000 7ff21fe00640 1 -- 172.21.15.57:0/1770189164 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7ff2140a0320 con 0x7ff2281049c0 2024-09-12T03:03:30.910 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:30.910 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"btime":"2024-09-12T03:03:22:546095+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:21.662051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24554},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24554,"qdb_cluster":[24554]},"id":1}]} 2024-09-12T03:03:30.910 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 17 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 msgr2=0x7ff20c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 0x7ff20c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff218009bd0 tx=0x7ff218005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 msgr2=0x7ff22819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 0x7ff22819ac00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e9f0 tx=0x7ff21400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 shutdown_connections 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff20c0a2b80 0x7ff20c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff228109140 0x7ff22819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2281049c0 0x7ff22819ac00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 --2- 172.21.15.57:0/1770189164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff228104010 0x7ff22819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:30.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 >> 172.21.15.57:0/1770189164 conn(0x7ff2280ffa50 msgr2=0x7ff228100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:30.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.911+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 shutdown_connections 2024-09-12T03:03:30.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:30.912+0000 7ff22e600640 1 -- 172.21.15.57:0/1770189164 wait complete. 2024-09-12T03:03:31.772 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:31 smithi057 ceph-mon[30361]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:31 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1770189164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:31.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:31 smithi204 ceph-mon[35013]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:31.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:31 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1770189164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:31 smithi181 ceph-mon[35320]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:31 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1770189164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:32.150 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 -- 172.21.15.57:0/2950859455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 msgr2=0x7f51641043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 --2- 172.21.15.57:0/2950859455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f51641043d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5154009900 tx=0x7f515402a410 comp rx=0 tx=0).stop 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 -- 172.21.15.57:0/2950859455 shutdown_connections 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 --2- 172.21.15.57:0/2950859455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 0x7f5164110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 --2- 172.21.15.57:0/2950859455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f51641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 --2- 172.21.15.57:0/2950859455 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f51641043d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 -- 172.21.15.57:0/2950859455 >> 172.21.15.57:0/2950859455 conn(0x7f51640ffa90 msgr2=0x7f5164101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 -- 172.21.15.57:0/2950859455 shutdown_connections 2024-09-12T03:03:32.761 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.760+0000 7f5169a00640 1 -- 172.21.15.57:0/2950859455 wait complete. 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.761+0000 7f5169a00640 1 Processor -- start 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.761+0000 7f5169a00640 1 -- start start 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f516419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f516419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 0x7f516419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.763 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5164114150 con 0x7f5164103ff0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5164113fd0 con 0x7f51641049a0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5169a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f51641142d0 con 0x7f5164109120 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5162a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f516419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5162a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f516419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39200/0 (socket says 172.21.15.57:39200) 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5162a00640 1 -- 172.21.15.57:0/3632429788 learned_addr learned my addr 172.21.15.57:0/3632429788 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f516419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f5163400640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 0x7f516419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 -- 172.21.15.57:0/3632429788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 msgr2=0x7f516419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 0x7f516419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 -- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 msgr2=0x7f516419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f516419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.762+0000 7f515be00640 1 -- 172.21.15.57:0/3632429788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419f5a0 con 0x7f51641049a0 2024-09-12T03:03:32.764 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f515be00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f516419ab30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f514c00e9f0 tx=0x7f514c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00ccc0 con 0x7f51641049a0 2024-09-12T03:03:32.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f514c010910 con 0x7f51641049a0 2024-09-12T03:03:32.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419f890 con 0x7f51641049a0 2024-09-12T03:03:32.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c018960 con 0x7f51641049a0 2024-09-12T03:03:32.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.763+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a7170 con 0x7f51641049a0 2024-09-12T03:03:32.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.764+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51641043d0 con 0x7f51641049a0 2024-09-12T03:03:32.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.766+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f514c010ac0 con 0x7f51641049a0 2024-09-12T03:03:32.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.767+0000 7f515b400640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.767+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f514c0cf670 con 0x7f51641049a0 2024-09-12T03:03:32.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.767+0000 7f5162a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.768+0000 7f5162a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 0x7f51480a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51540096c0 tx=0x7f5154005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.771+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f514c09bce0 con 0x7f51641049a0 2024-09-12T03:03:32.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.960+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f516402d7c0 con 0x7f51641049a0 2024-09-12T03:03:32.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.962+0000 7f515b400640 1 -- 172.21.15.57:0/3632429788 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24199 (secure 0 0 0) 0x7f514c09ffe0 con 0x7f51641049a0 2024-09-12T03:03:32.964 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:32.964 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":92,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:21.661916+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 msgr2=0x7f51480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 0x7f51480a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f51540096c0 tx=0x7f5154005a20 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 msgr2=0x7f516419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f516419ab30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f514c00e9f0 tx=0x7f514c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 shutdown_connections 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5164109120 0x7f516419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51641049a0 0x7f516419ab30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 --2- 172.21.15.57:0/3632429788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5164103ff0 0x7f516419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 >> 172.21.15.57:0/3632429788 conn(0x7f51640ffa90 msgr2=0x7f5164100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 shutdown_connections 2024-09-12T03:03:32.966 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:32.965+0000 7f5169a00640 1 -- 172.21.15.57:0/3632429788 wait complete. 2024-09-12T03:03:33.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-12T03:03:08.679612+0000', 'modified': '2024-09-12T03:03:21.662051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24554}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6839/399579254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 399579254}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 399579254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24554, 'qdb_cluster': [24554]} 2024-09-12T03:03:33.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-12T03:03:33.791 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:03:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:33 smithi204 ceph-mon[35013]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:33.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:33 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3632429788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:33.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:33 smithi181 ceph-mon[35320]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:33.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:33 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3632429788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:34.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:33 smithi057 ceph-mon[30361]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:34.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:33 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3632429788' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:34.168 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:34.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 -- 172.21.15.57:0/1822478451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a41047e0 msgr2=0x7f29a4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 --2- 172.21.15.57:0/1822478451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a41047e0 0x7f29a4108830 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a410 comp rx=0 tx=0).stop 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 -- 172.21.15.57:0/1822478451 shutdown_connections 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 --2- 172.21.15.57:0/1822478451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a4108f60 0x7f29a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 --2- 172.21.15.57:0/1822478451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a41047e0 0x7f29a4108830 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 --2- 172.21.15.57:0/1822478451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.726+0000 7f29a9600640 1 -- 172.21.15.57:0/1822478451 >> 172.21.15.57:0/1822478451 conn(0x7f29a40ff910 msgr2=0x7f29a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.727+0000 7f29a9600640 1 -- 172.21.15.57:0/1822478451 shutdown_connections 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.727+0000 7f29a9600640 1 -- 172.21.15.57:0/1822478451 wait complete. 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.727+0000 7f29a9600640 1 Processor -- start 2024-09-12T03:03:34.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 -- start start 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a41047e0 0x7f29a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4113f80 con 0x7f29a4108f60 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4113e00 con 0x7f29a41047e0 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4114100 con 0x7f29a4103e30 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39216/0 (socket says 172.21.15.57:39216) 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a8c00640 1 -- 172.21.15.57:0/4260989162 learned_addr learned my addr 172.21.15.57:0/4260989162 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.728+0000 7f29a3400640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a41047e0 0x7f29a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 -- 172.21.15.57:0/4260989162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a41047e0 msgr2=0x7f29a419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a41047e0 0x7f29a419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 -- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 msgr2=0x7f29a419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 -- 172.21.15.57:0/4260989162 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a419f460 con 0x7f29a4103e30 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a8c00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a3e00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a419a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a0c00640 1 -- 172.21.15.57:0/4260989162 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400ccc0 con 0x7f29a4103e30 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a0c00640 1 -- 172.21.15.57:0/4260989162 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2994010910 con 0x7f29a4103e30 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a419f750 con 0x7f29a4103e30 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a0c00640 1 -- 172.21.15.57:0/4260989162 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994018960 con 0x7f29a4103e30 2024-09-12T03:03:34.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.729+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a7030 con 0x7f29a4103e30 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.732+0000 7f29a0c00640 1 -- 172.21.15.57:0/4260989162 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2994010ac0 con 0x7f29a4103e30 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 Processor -- start 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 -- start start 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 0x5651adf458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5651adeab0b0 con 0x5651adf454e0 2024-09-12T03:03:34.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651ade52ab0 con 0x5651adf506c0 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651adea7070 con 0x5651adf45e00 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 0x5651adf458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.733+0000 7f29a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58146/0 (socket says 172.21.15.57:58146) 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a3400640 1 -- 172.21.15.57:0/3419451703 learned_addr learned my addr 172.21.15.57:0/3419451703 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 -- 172.21.15.57:0/3419451703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 msgr2=0x5651adf50180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf50180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 -- 172.21.15.57:0/3419451703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 msgr2=0x5651adf52ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 -- 172.21.15.57:0/3419451703 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651ade5f4a0 con 0x5651adf454e0 2024-09-12T03:03:34.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a3400640 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:34.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 0x5651adf458c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f299c011760 tx=0x7f299c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58aebc950d3b047 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.734+0000 7f29a0c00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 0x7f29880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f29a0c00640 1 -- 172.21.15.57:0/4260989162 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f29940cf670 con 0x7f29a4103e30 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f2999200640 1 -- 172.21.15.57:0/3419451703 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c004d20 con 0x5651adf454e0 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f29a3e00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 0x7f29880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f2999200640 1 -- 172.21.15.57:0/3419451703 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c005020 con 0x5651adf454e0 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f2999200640 1 -- 172.21.15.57:0/3419451703 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c019630 con 0x5651adf454e0 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3419451703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 msgr2=0x5651adf458c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f29a3e00640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 0x7f29880a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29940986b0 tx=0x7f29940d5110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.735+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 0x5651adf458c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f299c011760 tx=0x7f299c011c20 comp rx=0 tx=0).stop 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3419451703 shutdown_connections 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf506c0 0x5651adf52ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3419451703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf454e0 0x5651adf458c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3419451703 >> 172.21.15.57:0/3419451703 conn(0x5651adf41a70 msgr2=0x5651adf41e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3419451703 shutdown_connections 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3419451703 wait complete. 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 -- start start 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.736+0000 7f29ad7dd740 1 Processor -- start 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5651adf53330 con 0x5651adf506c0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651ade33920 con 0x5651adf454e0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651adf534b0 con 0x5651adf45e00 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29ad7dd740 10 client.0 _ll_register_callbacks cb 0x5651ade578a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf4d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58162/0 (socket says 172.21.15.57:58162) 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29a8c00640 1 -- 172.21.15.57:0/1092755869 learned_addr learned my addr 172.21.15.57:0/1092755869 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29a3400640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.737+0000 7f29a8c00640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 -- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 msgr2=0x5651adf4d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf4d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 -- 172.21.15.57:0/1092755869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 msgr2=0x5651adf4df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 -- 172.21.15.57:0/1092755869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651adc1c2b0 con 0x5651adf45e00 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29a3400640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4da00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f2990031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f7a5881f4b1981 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f298d600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990035040 con 0x5651adf45e00 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.738+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f299c005020 con 0x5651adf45e00 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2990004440 con 0x5651adf45e00 2024-09-12T03:03:34.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2990005040 con 0x5651adf45e00 2024-09-12T03:03:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f29ad7dd740 10 client.24652 fetch_fsmap learned FSMap version 17 2024-09-12T03:03:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5651adf5ef50 con 0x5651adf45e00 2024-09-12T03:03:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f299c005020 con 0x5651adf45e00 2024-09-12T03:03:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f29900035f0 con 0x5651adf45e00 2024-09-12T03:03:34.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.739+0000 7f29ad7dd740 10 client.24652 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T03:03:34.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29ad7dd740 10 client.24652 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:03:34.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29ad7dd740 20 client.24652 populate_metadata read hostname 'smithi057' 2024-09-12T03:03:34.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adf61520 0x5651adf62990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29ad7dd740 4 client.24652 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:34.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651adeb8620 con 0x5651adf61520 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.740+0000 7f29a8c00640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adf61520 0x5651adf62990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.741+0000 7f29a8c00640 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adf61520 0x5651adf62990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b92b8ad0d130d1 server_cookie=7e2a430d18891410 in_seq=0 out_seq=0 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.741+0000 7f298d600640 10 client.24652 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.741+0000 7f298d600640 1 -- 172.21.15.57:0/1092755869 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651adeb8620 con 0x5651adf61520 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f298d600640 10 client.24652 handle_command_reply: tid=0 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 client.24652 shutdown 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 msgr2=0x5651adf4da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4da00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f2990031850 comp rx=0 tx=0).stop 2024-09-12T03:03:34.747 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 shutdown_connections 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adf61520 0x5651adf62990 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4da00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/1092755869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf4d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f298d600640 0 client.24652 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 >> 172.21.15.57:0/1092755869 conn(0x5651adf41a70 msgr2=0x5651adf47750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 shutdown_connections 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 1 -- 172.21.15.57:0/1092755869 wait complete. 2024-09-12T03:03:34.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.742+0000 7f29ad7dd740 20 client.24652 trim_cache size 0 max 16384 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 Processor -- start 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 -- start start 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf3c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf506c0 0x5651adf4dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5651adf60cd0 con 0x5651adf45e00 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651adf613a0 con 0x5651adf454e0 2024-09-12T03:03:34.765 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651adf61a70 con 0x5651adf506c0 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.761+0000 7f29a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39242/0 (socket says 172.21.15.57:39242) 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 -- 172.21.15.57:0/355176444 learned_addr learned my addr 172.21.15.57:0/355176444 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a8c00640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf506c0 0x5651adf4dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a8c00640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf3c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 -- 172.21.15.57:0/355176444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf506c0 msgr2=0x5651adf4dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf506c0 0x5651adf4dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 -- 172.21.15.57:0/355176444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 msgr2=0x5651adf3c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf3c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 -- 172.21.15.57:0/355176444 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651adf4ba10 con 0x5651adf45e00 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a8c00640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf3c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f29a3400640 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2990003b30 tx=0x7f2990031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e186164cce2f5114 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f298fe00640 1 -- 172.21.15.57:0/355176444 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990035030 con 0x5651adf45e00 2024-09-12T03:03:34.766 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.762+0000 7f298fe00640 1 -- 172.21.15.57:0/355176444 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2990041070 con 0x5651adf45e00 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f298fe00640 1 -- 172.21.15.57:0/355176444 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990000e60 con 0x5651adf45e00 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 -- 172.21.15.57:0/355176444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 msgr2=0x5651adf4c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2990003b30 tx=0x7f2990031af0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 -- 172.21.15.57:0/355176444 shutdown_connections 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf506c0 0x5651adf4dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf45e00 0x5651adf4c4a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/355176444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf3c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 -- 172.21.15.57:0/355176444 >> 172.21.15.57:0/355176444 conn(0x5651adf41a70 msgr2=0x5651adf4bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 -- 172.21.15.57:0/355176444 shutdown_connections 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.763+0000 7f29ad7dd740 1 -- 172.21.15.57:0/355176444 wait complete. 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 -- start start 2024-09-12T03:03:34.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 Processor -- start 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5651adf62810 con 0x5651adf506c0 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5651adf62140 con 0x5651adf454e0 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651adf62ee0 con 0x5651adf45e00 2024-09-12T03:03:34.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29ad7dd740 10 client.0 _ll_register_callbacks cb 0x7f2990009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.764+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39252/0 (socket says 172.21.15.57:39252) 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 -- 172.21.15.57:0/3017250403 learned_addr learned my addr 172.21.15.57:0/3017250403 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a3400640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 -- 172.21.15.57:0/3017250403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 msgr2=0x5651adf4d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 -- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 msgr2=0x5651adf6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:34.771 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 -- 172.21.15.57:0/3017250403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651adf66ef0 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29a8c00640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5651adf4cc70 tx=0x7f299c02fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66912a9eeec7606a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f298d600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299c00fce0 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f299c0197b0 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.765+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2990041070 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f299c042050 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 10 client.14607 fetch_fsmap learned FSMap version 17 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5651adfcbd20 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2990041070 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f299c028900 con 0x5651adf506c0 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 10 client.14607 fetch_fsmap finished waiting for FSMap version 17 2024-09-12T03:03:34.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 10 client.14607 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 20 client.14607 populate_metadata read hostname 'smithi057' 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adfd0640 0x5651adfcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 4 client.14607 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.766+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "status"}) -- 0x5651adeb2b30 con 0x5651adfd0640 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.767+0000 7f29a3400640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adfd0640 0x5651adfcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.767+0000 7f29a3400640 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adfd0640 0x5651adfcf350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f10acaf63321786 server_cookie=e0adf2c875883e69 in_seq=0 out_seq=0 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.767+0000 7f298d600640 10 client.14607 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f298d600640 1 -- 172.21.15.57:0/3017250403 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5651adeb2b30 con 0x5651adfd0640 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f298d600640 10 client.14607 handle_command_reply: tid=0 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 client.14607 shutdown 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 msgr2=0x5651adf4db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5651adf4cc70 tx=0x7f299c02fe80 comp rx=0 tx=0).stop 2024-09-12T03:03:34.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 shutdown_connections 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5651adfd0640 0x5651adfcf350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5651adf506c0 0x5651adf4db20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5651adf45e00 0x5651adf4d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 --2- 172.21.15.57:0/3017250403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5651adf454e0 0x5651adf6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 >> 172.21.15.57:0/3017250403 conn(0x5651adf41a70 msgr2=0x5651adf5b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f298d600640 0 client.14607 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 shutdown_connections 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 1 -- 172.21.15.57:0/3017250403 wait complete. 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.768+0000 7f29ad7dd740 20 client.14607 trim_cache size 0 max 16384 2024-09-12T03:03:34.774 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24554, 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 25.317017113999999, 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 17, 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 92, 2024-09-12T03:03:34.775 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 92, 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 37.918089635000001 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 msgr2=0x7f29880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 0x7f29880a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29940986b0 tx=0x7f29940d5110 comp rx=0 tx=0).stop 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 msgr2=0x7f29a419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.776 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a419a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 shutdown_connections 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f29880a2b80 0x7f29880a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f29a4108f60 0x7f29a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29a41047e0 0x7f29a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 --2- 172.21.15.57:0/4260989162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29a4103e30 0x7f29a419a4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 >> 172.21.15.57:0/4260989162 conn(0x7f29a40ff910 msgr2=0x7f29a4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 shutdown_connections 2024-09-12T03:03:34.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:34.769+0000 7f29a9600640 1 -- 172.21.15.57:0/4260989162 wait complete. 2024-09-12T03:03:35.734 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:03:35.734 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:35.734 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:03:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:35 smithi204 ceph-mon[35013]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:35 smithi181 ceph-mon[35320]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:36.109 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:36.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:35 smithi057 ceph-mon[30361]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:36.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 -- 172.21.15.57:0/4174106193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 msgr2=0x7ffa14104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 --2- 172.21.15.57:0/4174106193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa14104210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009900 tx=0x7ffa0802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 -- 172.21.15.57:0/4174106193 shutdown_connections 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 --2- 172.21.15.57:0/4174106193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa14108f60 0x7ffa14110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 --2- 172.21.15.57:0/4174106193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa141047e0 0x7ffa14108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 --2- 172.21.15.57:0/4174106193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa14104210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 -- 172.21.15.57:0/4174106193 >> 172.21.15.57:0/4174106193 conn(0x7ffa140ff910 msgr2=0x7ffa14101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.730+0000 7ffa1a400640 1 -- 172.21.15.57:0/4174106193 shutdown_connections 2024-09-12T03:03:36.732 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.731+0000 7ffa1a400640 1 -- 172.21.15.57:0/4174106193 wait complete. 2024-09-12T03:03:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.731+0000 7ffa1a400640 1 Processor -- start 2024-09-12T03:03:36.733 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 -- start start 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 0x7ffa1419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14113fc0 con 0x7ffa14103e30 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14113e40 con 0x7ffa141047e0 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa1a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14114140 con 0x7ffa14108f60 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa19000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa19000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39276/0 (socket says 172.21.15.57:39276) 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa19000640 1 -- 172.21.15.57:0/3686005345 learned_addr learned my addr 172.21.15.57:0/3686005345 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa13e00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 0x7ffa1419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.732+0000 7ffa19a00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 -- 172.21.15.57:0/3686005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 msgr2=0x7ffa1419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 -- 172.21.15.57:0/3686005345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 msgr2=0x7ffa1419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 0x7ffa1419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 -- 172.21.15.57:0/3686005345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1419f440 con 0x7ffa14103e30 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19a00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa13e00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 0x7ffa1419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:36.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa19000640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa08006010 tx=0x7ffa08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.733+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0803f030 con 0x7ffa14103e30 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.734+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa08004440 con 0x7ffa14103e30 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.734+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1419f6d0 con 0x7ffa14103e30 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.734+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa08035070 con 0x7ffa14103e30 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.734+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a7010 con 0x7ffa14103e30 2024-09-12T03:03:36.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.735+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa14105310 con 0x7ffa14103e30 2024-09-12T03:03:36.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.737+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffa08042030 con 0x7ffa14103e30 2024-09-12T03:03:36.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.738+0000 7ffa11600640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 0x7ff9fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.738+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7ffa080ef620 con 0x7ffa14103e30 2024-09-12T03:03:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.738+0000 7ffa13e00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 0x7ff9fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.739+0000 7ffa13e00640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 0x7ff9fc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa140ffd30 tx=0x7ffa00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.742+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa080bbc90 con 0x7ffa14103e30 2024-09-12T03:03:36.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.968+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa1410c140 con 0x7ffa14103e30 2024-09-12T03:03:36.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.968+0000 7ffa11600640 1 -- 172.21.15.57:0/3686005345 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7ffa080bff90 con 0x7ffa14103e30 2024-09-12T03:03:36.969 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:03:36.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 msgr2=0x7ff9fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 0x7ff9fc0a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa140ffd30 tx=0x7ffa00005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:36.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 msgr2=0x7ffa1419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffa08006010 tx=0x7ffa08004040 comp rx=0 tx=0).stop 2024-09-12T03:03:36.971 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 shutdown_connections 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff9fc0a2b80 0x7ff9fc0a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa14108f60 0x7ffa1419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa141047e0 0x7ffa1419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 --2- 172.21.15.57:0/3686005345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa14103e30 0x7ffa1419a490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.970+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 >> 172.21.15.57:0/3686005345 conn(0x7ffa140ff910 msgr2=0x7ffa14100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.971+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 shutdown_connections 2024-09-12T03:03:36.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:36.971+0000 7ffa1a400640 1 -- 172.21.15.57:0/3686005345 wait complete. 2024-09-12T03:03:37.636 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-12T03:03:37.636 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:03:37.637 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:03:37.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:37.637 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:03:37.673 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42181sec preferred_lft 42181sec 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:03:37.674 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2024-09-12T03:03:37.675 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:03:37.675 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:03:37.675 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:03:37.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:03:37.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2024-09-12T03:03:37.675 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2024-09-12T03:03:37.676 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:03:37.676 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:03:37.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:03:37.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:37 smithi057 ceph-mon[30361]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:37.849 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:37 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3686005345' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:03:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:37 smithi204 ceph-mon[35013]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:37 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3686005345' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:03:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:37 smithi181 ceph-mon[35320]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:37 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3686005345' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:03:38.078 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:38.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:38.087 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:03:38.143 INFO:teuthology.orchestra.run.smithi057.stdout:1 2024-09-12T03:03:38.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:38.146 DEBUG:teuthology.orchestra.run.smithi057:> route 2024-09-12T03:03:38.201 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2024-09-12T03:03:38.202 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:03:38.202 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:03:38.202 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:03:38.202 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:03:38.202 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:03:38.202 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:03:38.203 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:03:38.203 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:03:38.203 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:03:38.203 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:03:38.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:03:38.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:38.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:38.809 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-09-12T03:03:38.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:38.846 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2024-09-12T03:03:38.922 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:03:38.922 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:03:38.922 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:38.922 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-12T03:03:38.922 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:03:39.042 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:03:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:39.103 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-12T03:03:39.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-12T03:03:39.104 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-12T03:03:39.104 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:03:39.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:03:39.342 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:39.351 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:03:39.351 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:03:39.351 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2024-09-12T03:03:39.351 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:03:39.351 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:03:39.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:03:39.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:39.521 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-12T03:03:39.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:03:39.522 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:39.557 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-12T03:03:39.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:03:39.557 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:39.632 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-09-12T03:03:39.729 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:03:39.769 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-09-12T03:03:39.769 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:39.770 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:03:39.771 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/35d6d0a8854e7894f9c7af1aedb006d917166b2bdae670fd00aae3f0dd1a4877/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/848c498ae2794dba55e502fbb8293c99d2d8ef4011257ab71234707d7d8a4034/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/554fa3d6725f8d33ddba6d944c3d8637c66b2017b4cb9c6cd39ad3a94cf1cfad/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f18dea659697ae2ad35267901725b89b48720df684bd56119ca28ac4c9b3f787/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/51a5c090152be0a86bc8dafba3c46aa95c8a72906f6fe3f3203572cb58d51647/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/4bcbf9afdae155732d1ca8346a028dbffdeb1e72b0cf510311231d366ad3dcc6/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e526d504b10826f44ec507f524ab0dca8c44668aa847a9caf2d915b50211b6d0/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e4ced4bffb4d9632334bc40b764c1a1ab50fc8f3ee060e1a13387e709cbfec95/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bf709dc3f808cde7ddf736b45cbd03a1a1f15c935f85441721d72324709229b4/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f778c3edf7f41c54f959522f983e3be7b9ab7cf08924e33f93ff92ed466e4152/merged 2024-09-12T03:03:39.772 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-09-12T03:03:39.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:39.773 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:39.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:39.774 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-09-12T03:03:39.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-12T03:03:39.850 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-12T03:03:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:39 smithi204 ceph-mon[35013]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:39.894 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-09-12T03:03:39.941 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:39 smithi057 ceph-mon[30361]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:39.952 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:03:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:39 smithi181 ceph-mon[35320]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:39.990 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-09-12T03:03:39.990 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-09-12T03:03:39.990 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-09-12T03:03:39.990 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-09-12T03:03:39.990 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-09-12T03:03:39.991 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/35d6d0a8854e7894f9c7af1aedb006d917166b2bdae670fd00aae3f0dd1a4877/merged 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/848c498ae2794dba55e502fbb8293c99d2d8ef4011257ab71234707d7d8a4034/merged 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-09-12T03:03:39.992 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/554fa3d6725f8d33ddba6d944c3d8637c66b2017b4cb9c6cd39ad3a94cf1cfad/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f18dea659697ae2ad35267901725b89b48720df684bd56119ca28ac4c9b3f787/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/51a5c090152be0a86bc8dafba3c46aa95c8a72906f6fe3f3203572cb58d51647/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/4bcbf9afdae155732d1ca8346a028dbffdeb1e72b0cf510311231d366ad3dcc6/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e526d504b10826f44ec507f524ab0dca8c44668aa847a9caf2d915b50211b6d0/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e4ced4bffb4d9632334bc40b764c1a1ab50fc8f3ee060e1a13387e709cbfec95/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bf709dc3f808cde7ddf736b45cbd03a1a1f15c935f85441721d72324709229b4/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f778c3edf7f41c54f959522f983e3be7b9ab7cf08924e33f93ff92ed466e4152/merged 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:39.993 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:39.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:39.994 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-09-12T03:03:40.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:2024-09-12T03:03:40.068+0000 7fd74780a480 -1 init, newargv = 0x55bad0647430 newargc=15 2024-09-12T03:03:40.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[74411]: starting ceph client 2024-09-12T03:03:40.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[74411]: starting fuse 2024-09-12T03:03:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-12T03:03:40.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T03:03:40.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:40 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-12T03:03:41.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:03:41.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T03:03:41.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T03:03:41.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:40 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:03:41.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-12T03:03:41.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-12T03:03:41.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-09-12T03:03:41.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:40 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-09-12T03:03:41.172 DEBUG:teuthology.orchestra.run.smithi057:> sudo modprobe fuse 2024-09-12T03:03:41.237 DEBUG:teuthology.orchestra.run.smithi057:> cat /proc/self/mounts | awk '{print $2}' 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/proc 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/sys 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/dev 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/security 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/shm 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/pts 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/run 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/cgroup 2024-09-12T03:03:41.276 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/pstore 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/bpf 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/ 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/selinux 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/hugepages 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/dev/mqueue 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/debug 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/tracing 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/fs/fuse/connections 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-sysctl.service 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/sys/kernel/config 2024-09-12T03:03:41.277 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/nfs/rpc_pipefs 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/1000 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/ceph 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/35d6d0a8854e7894f9c7af1aedb006d917166b2bdae670fd00aae3f0dd1a4877/merged 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/848c498ae2794dba55e502fbb8293c99d2d8ef4011257ab71234707d7d8a4034/merged 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/run/user/0 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/proc/sys/fs/binfmt_misc 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/554fa3d6725f8d33ddba6d944c3d8637c66b2017b4cb9c6cd39ad3a94cf1cfad/merged 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f18dea659697ae2ad35267901725b89b48720df684bd56119ca28ac4c9b3f787/merged 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/51a5c090152be0a86bc8dafba3c46aa95c8a72906f6fe3f3203572cb58d51647/merged 2024-09-12T03:03:41.278 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/4bcbf9afdae155732d1ca8346a028dbffdeb1e72b0cf510311231d366ad3dcc6/merged 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e526d504b10826f44ec507f524ab0dca8c44668aa847a9caf2d915b50211b6d0/merged 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/e4ced4bffb4d9632334bc40b764c1a1ab50fc8f3ee060e1a13387e709cbfec95/merged 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/bf709dc3f808cde7ddf736b45cbd03a1a1f15c935f85441721d72324709229b4/merged 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/var/lib/containers/storage/overlay/f778c3edf7f41c54f959522f983e3be7b9ab7cf08924e33f93ff92ed466e4152/merged 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:03:41.279 INFO:teuthology.orchestra.run.smithi057.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:41.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:41.280 DEBUG:teuthology.orchestra.run.smithi057:> ls /sys/fs/fuse/connections 2024-09-12T03:03:41.357 INFO:teuthology.orchestra.run.smithi057.stdout:108 2024-09-12T03:03:41.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [108] 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> sudo stdin-killer -- python3 -c ' 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> import glob 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> import re 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> import os 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> import subprocess 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> def _find_admin_socket(client_name): 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> files = glob.glob(asok_path) 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> 2024-09-12T03:03:41.358 DEBUG:teuthology.orchestra.run.smithi057:> # Given a non-glob path, it better be there 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> if "*" not in asok_path: 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> assert(len(files) == 1) 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> return files[0] 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> for f in files: 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> contents = proc_f.read() 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> if mountpoint in contents: 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> return f 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> 2024-09-12T03:03:41.359 DEBUG:teuthology.orchestra.run.smithi057:> print(_find_admin_socket("client.admin")) 2024-09-12T03:03:41.360 DEBUG:teuthology.orchestra.run.smithi057:> ' 2024-09-12T03:03:41.522 INFO:teuthology.orchestra.run.smithi057.stdout:/var/run/ceph/ceph-client.admin.74411.asok 2024-09-12T03:03:41.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:41.538 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74411.asok 2024-09-12T03:03:41.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:41.538 DEBUG:teuthology.orchestra.run.smithi057:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74411.asok status 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout: "metadata": { 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout: "entity_id": "admin", 2024-09-12T03:03:41.727 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": "smithi057", 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "pid": "74411", 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "root": "/" 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_count": 0, 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "dentry_pinned_count": 0, 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 14619, 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "inst": { 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": { 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "client", 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: "num": 14619 2024-09-12T03:03:41.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3978648417 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": { 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "192.168.0.1:0", 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3978648417 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:03:41.729 INFO:teuthology.orchestra.run.smithi057.stdout: "inst_str": "client.14619 192.168.0.1:0/3978648417", 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "addr_str": "192.168.0.1:0/3978648417", 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "inode_count": 1, 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_epoch": 17, 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch": 92, 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_epoch_barrier": 0, 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "blocklisted": false, 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs" 2024-09-12T03:03:41.730 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:03:41.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:41.740 DEBUG:teuthology.orchestra.run.smithi057:> stat --file-system '--printf=%T 2024-09-12T03:03:41.740 DEBUG:teuthology.orchestra.run.smithi057:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:41.776 INFO:teuthology.orchestra.run.smithi057.stdout:fuseblk 2024-09-12T03:03:41.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:41.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:41.776 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:41.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:41.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-12T03:03:41.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:03:41.865 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:41 smithi057 ceph-mon[30361]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:03:41.874 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:03:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:03:41.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:41 smithi181 ceph-mon[35320]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:03:41.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:03:42.007 INFO:teuthology.orchestra.run.smithi057.stdout:# file: . 2024-09-12T03:03:42.007 INFO:teuthology.orchestra.run.smithi057.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-12T03:03:42.007 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:42.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:03:42.016 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:03:42.016 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:03:42.052 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:03:42.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-09-12T03:03:42.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:41 smithi204 ceph-mon[35013]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T03:03:42.474 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:42.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:42 smithi057 ceph-mon[30361]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:03:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:42 smithi181 ceph-mon[35320]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:03:43.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:42 smithi204 ceph-mon[35013]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 -- 172.21.15.57:0/2841499749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc103ff0 msgr2=0x7fd1cc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 --2- 172.21.15.57:0/2841499749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc1043d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009900 tx=0x7fd1bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 -- 172.21.15.57:0/2841499749 shutdown_connections 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 --2- 172.21.15.57:0/2841499749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc109120 0x7fd1cc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 --2- 172.21.15.57:0/2841499749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 --2- 172.21.15.57:0/2841499749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc1043d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 -- 172.21.15.57:0/2841499749 >> 172.21.15.57:0/2841499749 conn(0x7fd1cc0ffa90 msgr2=0x7fd1cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 -- 172.21.15.57:0/2841499749 shutdown_connections 2024-09-12T03:03:43.310 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.285+0000 7fd1d1600640 1 -- 172.21.15.57:0/2841499749 wait complete. 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.286+0000 7fd1d1600640 1 Processor -- start 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.286+0000 7fd1d1600640 1 -- start start 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc109120 0x7fd1cc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc114190 con 0x7fd1cc1049a0 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc114010 con 0x7fd1cc103ff0 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1d1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc114310 con 0x7fd1cc109120 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42104/0 (socket says 172.21.15.57:42104) 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 -- 172.21.15.57:0/838793568 learned_addr learned my addr 172.21.15.57:0/838793568 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1cb400640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc109120 0x7fd1cc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1caa00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 -- 172.21.15.57:0/838793568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc109120 msgr2=0x7fd1cc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc109120 0x7fd1cc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 -- 172.21.15.57:0/838793568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 msgr2=0x7fd1cc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1c3e00640 1 -- 172.21.15.57:0/838793568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc19f640 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.287+0000 7fd1caa00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1c3e00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e9f0 tx=0x7fd1b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400cd80 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b4005500 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc19f930 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4010690 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.288+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a7210 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.289+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc1043d0 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.292+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd1b40046c0 con 0x7fd1cc1049a0 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.293+0000 7fd1c3400640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 0x7fd1b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.293+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fd1b40cf890 con 0x7fd1cc1049a0 2024-09-12T03:03:43.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.294+0000 7fd1caa00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 0x7fd1b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.294+0000 7fd1caa00640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 0x7fd1b00a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005ab0 tx=0x7fd1bc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.298+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1b409c430 con 0x7fd1cc1049a0 2024-09-12T03:03:43.496 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.496+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1cc10bd70 con 0x7fd1cc1049a0 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.496+0000 7fd1c3400640 1 -- 172.21.15.57:0/838793568 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+589 (secure 0 0 0) 0x7fd1b409bd30 con 0x7fd1cc1049a0 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1578458239 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1345986708 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:43.497 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1176788560 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:43.498 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:43.498 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/4021182045 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:43.498 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/752811668 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:43.498 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1256058423 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:43.498 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-09-12T03:03:43.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 msgr2=0x7fd1b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 0x7fd1b00a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005ab0 tx=0x7fd1bc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 msgr2=0x7fd1cc19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400e9f0 tx=0x7fd1b400edf0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 shutdown_connections 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd1b00a2b80 0x7fd1b00a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1cc109120 0x7fd1cc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1cc1049a0 0x7fd1cc19abd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 --2- 172.21.15.57:0/838793568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1cc103ff0 0x7fd1cc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.499+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 >> 172.21.15.57:0/838793568 conn(0x7fd1cc0ffa90 msgr2=0x7fd1cc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.500+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 shutdown_connections 2024-09-12T03:03:43.501 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:43.500+0000 7fd1d1600640 1 -- 172.21.15.57:0/838793568 wait complete. 2024-09-12T03:03:43.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:43 smithi057 ceph-mon[30361]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:43.856 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:43 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/838793568' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:43 smithi181 ceph-mon[35320]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:43 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/838793568' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:44.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:43 smithi204 ceph-mon[35013]: pgmap v325: 129 pgs: 1 peering, 128 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:03:44.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:43 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/838793568' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:44.308 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:03:44.308 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:03:44.345 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:03:44.768 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:45.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 -- 172.21.15.57:0/686947331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d01049c0 msgr2=0x7f55d0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 --2- 172.21.15.57:0/686947331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d01049c0 0x7f55d0108a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009900 tx=0x7f55bc02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 -- 172.21.15.57:0/686947331 shutdown_connections 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 --2- 172.21.15.57:0/686947331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0109140 0x7f55d0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 --2- 172.21.15.57:0/686947331 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d01049c0 0x7f55d0108a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 --2- 172.21.15.57:0/686947331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d0104010 0x7f55d01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 -- 172.21.15.57:0/686947331 >> 172.21.15.57:0/686947331 conn(0x7f55d00ffa50 msgr2=0x7f55d0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 -- 172.21.15.57:0/686947331 shutdown_connections 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.346+0000 7f55d5400640 1 -- 172.21.15.57:0/686947331 wait complete. 2024-09-12T03:03:45.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 Processor -- start 2024-09-12T03:03:45.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 -- start start 2024-09-12T03:03:45.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 0x7f55d019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 0x7f55d019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d0109140 0x7f55d019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f55d0114140 con 0x7f55d0109140 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f55d0113fc0 con 0x7f55d01049c0 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.347+0000 7f55d5400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55d01142c0 con 0x7f55d0104010 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55ce800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 0x7f55d019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55ce800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 0x7f55d019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:37244/0 (socket says 172.21.15.57:37244) 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 0x7f55d019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55ce800640 1 -- 172.21.15.57:0/1322136579 learned_addr learned my addr 172.21.15.57:0/1322136579 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cf200640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d0109140 0x7f55d019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 -- 172.21.15.57:0/1322136579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 msgr2=0x7f55d019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 0x7f55d019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.350 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 -- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d0109140 msgr2=0x7f55d019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d0109140 0x7f55d019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.348+0000 7f55cde00640 1 -- 172.21.15.57:0/1322136579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d019f650 con 0x7f55d01049c0 2024-09-12T03:03:45.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55cde00640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 0x7f55d019abe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005940 tx=0x7f55bc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc03f030 con 0x7f55d01049c0 2024-09-12T03:03:45.351 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc0044c0 con 0x7f55d01049c0 2024-09-12T03:03:45.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d019f8e0 con 0x7f55d01049c0 2024-09-12T03:03:45.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc035070 con 0x7f55d01049c0 2024-09-12T03:03:45.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.349+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d01a7220 con 0x7f55d01049c0 2024-09-12T03:03:45.352 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.351+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d01043f0 con 0x7f55d01049c0 2024-09-12T03:03:45.353 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.352+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55bc042030 con 0x7f55d01049c0 2024-09-12T03:03:45.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.354+0000 7f55c7400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 0x7f55b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.354 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.354+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f55bc0ef630 con 0x7f55d01049c0 2024-09-12T03:03:45.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.354+0000 7f55ce800640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 0x7f55b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.355+0000 7f55ce800640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 0x7f55b40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009c90 tx=0x7f55c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.358 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.358+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55bc0bc1d0 con 0x7f55d01049c0 2024-09-12T03:03:45.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.557+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55d010c380 con 0x7f55d01049c0 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.557+0000 7f55c7400640 1 -- 172.21.15.57:0/1322136579 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+589 (secure 0 0 0) 0x7f55bc0bbad0 con 0x7f55d01049c0 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1578458239 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1345986708 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:45.558 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1176788560 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:45.559 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:45.559 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/4021182045 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:45.559 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/752811668 2024-09-13T02:55:39.619488+0000 2024-09-12T03:03:45.559 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1256058423 2024-09-13T02:55:10.466284+0000 2024-09-12T03:03:45.559 INFO:teuthology.orchestra.run.smithi057.stderr:listed 10 entries 2024-09-12T03:03:45.560 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 msgr2=0x7f55b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 0x7f55b40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009c90 tx=0x7f55c0005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 msgr2=0x7f55d019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 0x7f55d019abe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005940 tx=0x7f55bc0040c0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 shutdown_connections 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f55b40a2c00 0x7f55b40a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f55d0109140 0x7f55d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55d01049c0 0x7f55d019abe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 --2- 172.21.15.57:0/1322136579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55d0104010 0x7f55d019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 >> 172.21.15.57:0/1322136579 conn(0x7f55d00ffa50 msgr2=0x7f55d0100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 shutdown_connections 2024-09-12T03:03:45.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:45.560+0000 7f55d5400640 1 -- 172.21.15.57:0/1322136579 wait complete. 2024-09-12T03:03:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:45 smithi057 ceph-mon[30361]: pgmap v326: 129 pgs: 2 activating, 1 peering, 126 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:45 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1322136579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:45 smithi181 ceph-mon[35320]: pgmap v326: 129 pgs: 2 activating, 1 peering, 126 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:45 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1322136579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:46.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:45 smithi204 ceph-mon[35013]: pgmap v326: 129 pgs: 2 activating, 1 peering, 126 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:46.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:45 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1322136579' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:03:46.304 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi057.front.sepia.ceph.com... 2024-09-12T03:03:46.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:46.304 DEBUG:teuthology.orchestra.run.smithi057:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-12T03:03:46.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi057.stderr:ceph-fuse[74411]: fuse finished with error 0 and tester_r 0 2024-09-12T03:03:46.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T03:03:46.452 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:03:46.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:03:46.453 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-12T03:03:46.500 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:46.921 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:47.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 -- 172.21.15.57:0/3591719369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 msgr2=0x7f41f410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:47.534 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 --2- 172.21.15.57:0/3591719369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f410ea90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009ca0 tx=0x7f41ec02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 -- 172.21.15.57:0/3591719369 shutdown_connections 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 --2- 172.21.15.57:0/3591719369 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f410ea90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 --2- 172.21.15.57:0/3591719369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4102790 0x7f41f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 --2- 172.21.15.57:0/3591719369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4101de0 0x7f41f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.533+0000 7f41faa00640 1 -- 172.21.15.57:0/3591719369 >> 172.21.15.57:0/3591719369 conn(0x7f41f40fd8c0 msgr2=0x7f41f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.534+0000 7f41faa00640 1 -- 172.21.15.57:0/3591719369 shutdown_connections 2024-09-12T03:03:47.535 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.534+0000 7f41faa00640 1 -- 172.21.15.57:0/3591719369 wait complete. 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.534+0000 7f41faa00640 1 Processor -- start 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 -- start start 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4101de0 0x7f41f419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 0x7f41f419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4111f70 con 0x7f41f4106f10 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f41f4111df0 con 0x7f41f4102790 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41faa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41f41120f0 con 0x7f41f4101de0 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41f8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41f8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42130/0 (socket says 172.21.15.57:42130) 2024-09-12T03:03:47.537 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41f8a00640 1 -- 172.21.15.57:0/2700748085 learned_addr learned my addr 172.21.15.57:0/2700748085 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41f3400640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 0x7f41f419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.535+0000 7f41f3e00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4101de0 0x7f41f419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 -- 172.21.15.57:0/2700748085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4101de0 msgr2=0x7f41f419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4101de0 0x7f41f419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 -- 172.21.15.57:0/2700748085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 msgr2=0x7f41f419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 0x7f41f419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 -- 172.21.15.57:0/2700748085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f419f5c0 con 0x7f41f4106f10 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f3400640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 0x7f41f419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:47.538 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.536+0000 7f41f8a00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f41ec031a20 tx=0x7f41ec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.537+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec003b70 con 0x7f41f4106f10 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.537+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41ec003d10 con 0x7f41f4106f10 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.537+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec02fc40 con 0x7f41f4106f10 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.537+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f419f850 con 0x7f41f4106f10 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.537+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41a7190 con 0x7f41f4106f10 2024-09-12T03:03:47.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.538+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f41021c0 con 0x7f41f4106f10 2024-09-12T03:03:47.540 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.540+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f41ec004050 con 0x7f41f4106f10 2024-09-12T03:03:47.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.541+0000 7f41f0c00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 0x7f41d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:47.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.541+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f41ec0f0490 con 0x7f41f4106f10 2024-09-12T03:03:47.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.541+0000 7f41f3e00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 0x7f41d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:47.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.542+0000 7f41f3e00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 0x7f41d80a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009bd0 tx=0x7f41e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:47.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.545+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41ec0bd0c0 con 0x7f41f4106f10 2024-09-12T03:03:47.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.733+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41f41a7430 con 0x7f41f4106f10 2024-09-12T03:03:47.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.735+0000 7f41f0c00640 1 -- 172.21.15.57:0/2700748085 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f41ec0bc9c0 con 0x7f41f4106f10 2024-09-12T03:03:47.736 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:47.736 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:41.651986+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:47.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.737+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 msgr2=0x7f41d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:47.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.737+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 0x7f41d80a50c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009bd0 tx=0x7f41e4005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:47.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.737+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 msgr2=0x7f41f419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:47.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.737+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f41ec031a20 tx=0x7f41ec031a50 comp rx=0 tx=0).stop 2024-09-12T03:03:47.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 shutdown_connections 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f41d80a2c00 0x7f41d80a50c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f41f4106f10 0x7f41f419eee0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41f4102790 0x7f41f419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 --2- 172.21.15.57:0/2700748085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41f4101de0 0x7f41f419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 >> 172.21.15.57:0/2700748085 conn(0x7f41f40fd8c0 msgr2=0x7f41f40ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 shutdown_connections 2024-09-12T03:03:47.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:47.738+0000 7f41faa00640 1 -- 172.21.15.57:0/2700748085 wait complete. 2024-09-12T03:03:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:47 smithi181 ceph-mon[35320]: pgmap v327: 129 pgs: 2 activating, 1 peering, 126 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:47 smithi057 ceph-mon[30361]: pgmap v327: 129 pgs: 2 activating, 1 peering, 126 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:47 smithi204 ceph-mon[35013]: pgmap v327: 129 pgs: 2 activating, 1 peering, 126 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:48.518 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:03:48.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2700748085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:48.906 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2700748085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:49.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2700748085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:49.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 -- 172.21.15.57:0/2292100036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628101de0 msgr2=0x7f06281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:49.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 --2- 172.21.15.57:0/2292100036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628101de0 0x7f06281021c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02a400 comp rx=0 tx=0).stop 2024-09-12T03:03:49.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 -- 172.21.15.57:0/2292100036 shutdown_connections 2024-09-12T03:03:49.438 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 --2- 172.21.15.57:0/2292100036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628106f10 0x7f062810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 --2- 172.21.15.57:0/2292100036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628102790 0x7f06281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 --2- 172.21.15.57:0/2292100036 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628101de0 0x7f06281021c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 -- 172.21.15.57:0/2292100036 >> 172.21.15.57:0/2292100036 conn(0x7f06280fd8c0 msgr2=0x7f06280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:49.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 -- 172.21.15.57:0/2292100036 shutdown_connections 2024-09-12T03:03:49.439 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.437+0000 7f0630400640 1 -- 172.21.15.57:0/2292100036 wait complete. 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.438+0000 7f0630400640 1 Processor -- start 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.438+0000 7f0630400640 1 -- start start 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 0x7f06280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 0x7f0628074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0628111f80 con 0x7f0628106f10 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0628111e00 con 0x7f0628101de0 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f0630400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0628112100 con 0x7f0628102790 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f062d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:49.440 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f062da00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 0x7f06280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:49.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f062e400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 0x7f0628074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:49.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f062d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56434/0 (socket says 172.21.15.57:56434) 2024-09-12T03:03:49.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.439+0000 7f062d000640 1 -- 172.21.15.57:0/1966644071 learned_addr learned my addr 172.21.15.57:0/1966644071 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:49.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 -- 172.21.15.57:0/1966644071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 msgr2=0x7f0628074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:49.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 -- 172.21.15.57:0/1966644071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 msgr2=0x7f06280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 0x7f06280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 -- 172.21.15.57:0/1966644071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06281a6920 con 0x7f0628106f10 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062da00640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 0x7f06280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062d000640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f062e400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 0x7f0628074f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f062000e940 tx=0x7f062000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.440+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620004d20 con 0x7f0628106f10 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.441+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0620005020 con 0x7f0628106f10 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.441+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200116b0 con 0x7f0628106f10 2024-09-12T03:03:49.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.441+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06281a6c10 con 0x7f0628106f10 2024-09-12T03:03:49.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.441+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a7100 con 0x7f0628106f10 2024-09-12T03:03:49.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.442+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06281021c0 con 0x7f0628106f10 2024-09-12T03:03:49.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.444+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f06200054d0 con 0x7f0628106f10 2024-09-12T03:03:49.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.445+0000 7f0626000640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 0x7f06100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:49.445 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.445+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f06200cf3b0 con 0x7f0628106f10 2024-09-12T03:03:49.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.445+0000 7f062da00640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 0x7f06100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:49.446 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.446+0000 7f062da00640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 0x7f06100a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:49.449 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.449+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0620016030 con 0x7f0628106f10 2024-09-12T03:03:49.638 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.638+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0628109b60 con 0x7f0628106f10 2024-09-12T03:03:49.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.639+0000 7f0626000640 1 -- 172.21.15.57:0/1966644071 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f062009b900 con 0x7f0628106f10 2024-09-12T03:03:49.641 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:03:49.641 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:41.651986+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:03:49.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 msgr2=0x7f06100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:49.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 0x7f06100a50c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f061c000c00 tx=0x7f061c031510 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 msgr2=0x7f0628074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 0x7f0628074f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f062000e940 tx=0x7f062000ee00 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 shutdown_connections 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f06100a2c00 0x7f06100a50c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0628106f10 0x7f0628074f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0628102790 0x7f0628074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 --2- 172.21.15.57:0/1966644071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0628101de0 0x7f06280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 >> 172.21.15.57:0/1966644071 conn(0x7f06280fd8c0 msgr2=0x7f06280fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 shutdown_connections 2024-09-12T03:03:49.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:49.642+0000 7f0630400640 1 -- 172.21.15.57:0/1966644071 wait complete. 2024-09-12T03:03:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:49 smithi181 ceph-mon[35320]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:49 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1966644071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:50.061 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:49 smithi057 ceph-mon[30361]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:50.061 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:49 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1966644071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:49 smithi204 ceph-mon[35013]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:49 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1966644071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:03:50.443 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:03:50.821 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:51.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:50 smithi057 ceph-mon[30361]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 451 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:51.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:50 smithi204 ceph-mon[35013]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 451 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:51.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:50 smithi181 ceph-mon[35320]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 451 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 -- 172.21.15.57:0/1418896688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a0109120 msgr2=0x7f80a0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 --2- 172.21.15.57:0/1418896688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a0109120 0x7f80a0110ca0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8098009ca0 tx=0x7f809802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 -- 172.21.15.57:0/1418896688 shutdown_connections 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 --2- 172.21.15.57:0/1418896688 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a0109120 0x7f80a0110ca0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 --2- 172.21.15.57:0/1418896688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a01049a0 0x7f80a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 --2- 172.21.15.57:0/1418896688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 0x7f80a01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 -- 172.21.15.57:0/1418896688 >> 172.21.15.57:0/1418896688 conn(0x7f80a00ffa90 msgr2=0x7f80a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 -- 172.21.15.57:0/1418896688 shutdown_connections 2024-09-12T03:03:51.419 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.418+0000 7f80a7c00640 1 -- 172.21.15.57:0/1418896688 wait complete. 2024-09-12T03:03:51.420 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.419+0000 7f80a7c00640 1 Processor -- start 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.419+0000 7f80a7c00640 1 -- start start 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 0x7f80a019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a0109120 0x7f80a019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0114130 con 0x7f80a01049a0 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0113fb0 con 0x7f80a0103ff0 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a7c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01142b0 con 0x7f80a0109120 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59678/0 (socket says 172.21.15.57:59678) 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 -- 172.21.15.57:0/3310673148 learned_addr learned my addr 172.21.15.57:0/3310673148 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a5200640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 0x7f80a019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.421 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f80a5c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a0109120 0x7f80a019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 -- 172.21.15.57:0/3310673148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a0109120 msgr2=0x7f80a019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a0109120 0x7f80a019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.422 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 -- 172.21.15.57:0/3310673148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 msgr2=0x7f80a019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 0x7f80a019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.420+0000 7f809fe00640 1 -- 172.21.15.57:0/3310673148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a019f540 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f809fe00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00cd80 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c010910 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a019f830 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c018990 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.421+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01a7110 con 0x7f80a01049a0 2024-09-12T03:03:51.423 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.422+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a01043d0 con 0x7f80a01049a0 2024-09-12T03:03:51.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.425+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f808c010ac0 con 0x7f80a01049a0 2024-09-12T03:03:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.426+0000 7f809d600640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 0x7f80880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.426+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f808c0cfa20 con 0x7f80a01049a0 2024-09-12T03:03:51.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.427+0000 7f80a5200640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 0x7f80880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:51.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.427+0000 7f80a5200640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 0x7f80880a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8094009bd0 tx=0x7f8094005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:51.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.428+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f808c09bf70 con 0x7f80a01049a0 2024-09-12T03:03:51.652 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.652+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f80a019b8f0 con 0x7f80a01049a0 2024-09-12T03:03:51.768 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.768+0000 7f809d600640 1 -- 172.21.15.57:0/3310673148 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f80a019b8f0 con 0x7f80a01049a0 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 msgr2=0x7f80880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 0x7f80880a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8094009bd0 tx=0x7f8094005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 msgr2=0x7f80a019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 shutdown_connections 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f80880a2c00 0x7f80880a50c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.772 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80a0109120 0x7f80a019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f80a01049a0 0x7f80a019aad0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 --2- 172.21.15.57:0/3310673148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80a0103ff0 0x7f80a019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:51.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.771+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 >> 172.21.15.57:0/3310673148 conn(0x7f80a00ffa90 msgr2=0x7f80a0101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:51.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.772+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 shutdown_connections 2024-09-12T03:03:51.773 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:51.772+0000 7f80a7c00640 1 -- 172.21.15.57:0/3310673148 wait complete. 2024-09-12T03:03:52.104 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:51 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:03:52.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:51 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:03:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:51 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:03:52.627 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T03:03:52.627 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T03:03:52.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:52 smithi057 ceph-mon[30361]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 509 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:52.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:52 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:03:52.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:52 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:53.007 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:53.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:52 smithi204 ceph-mon[35013]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 509 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:53.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:52 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:03:53.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:52 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:53.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:52 smithi181 ceph-mon[35320]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 509 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:53.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:52 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3310673148' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:03:53.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:52 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:53.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.549+0000 7f4777600640 1 -- 172.21.15.57:0/1333679255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770103ff0 msgr2=0x7f47701043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.549+0000 7f4777600640 1 --2- 172.21.15.57:0/1333679255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770103ff0 0x7f47701043d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4764009900 tx=0x7f476402a410 comp rx=0 tx=0).stop 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 -- 172.21.15.57:0/1333679255 shutdown_connections 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 --2- 172.21.15.57:0/1333679255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770109120 0x7f4770110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 --2- 172.21.15.57:0/1333679255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f47701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 --2- 172.21.15.57:0/1333679255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770103ff0 0x7f47701043d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 -- 172.21.15.57:0/1333679255 >> 172.21.15.57:0/1333679255 conn(0x7f47700ffa90 msgr2=0x7f4770101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 -- 172.21.15.57:0/1333679255 shutdown_connections 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.550+0000 7f4777600640 1 -- 172.21.15.57:0/1333679255 wait complete. 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 Processor -- start 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 -- start start 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f477019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4770114170 con 0x7f4770109120 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4770113ff0 con 0x7f47701049a0 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4777600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47701142f0 con 0x7f4770103ff0 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.551+0000 7f4775600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f476fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f477019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4775600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59700/0 (socket says 172.21.15.57:59700) 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56466/0 (socket says 172.21.15.57:56466) 2024-09-12T03:03:53.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4775600640 1 -- 172.21.15.57:0/4010314773 learned_addr learned my addr 172.21.15.57:0/4010314773 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:53.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 -- 172.21.15.57:0/4010314773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 msgr2=0x7f477019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f477019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 -- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 msgr2=0x7f477019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f4774c00640 1 -- 172.21.15.57:0/4010314773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477019f620 con 0x7f4770103ff0 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.552+0000 7f476fe00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f477019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f4775600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:53.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f4774c00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f476402a930 tx=0x7f4764003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476403f070 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4764031c90 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477019f8b0 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764035030 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.553+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701a71f0 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.555+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47701054d0 con 0x7f4770103ff0 2024-09-12T03:03:53.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.557+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4764042070 con 0x7f4770103ff0 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.558+0000 7f476d600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:53.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.558+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f47640f03d0 con 0x7f4770103ff0 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.558+0000 7f476fe00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:53.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.559+0000 7f476fe00640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 0x7f47580a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47700fff40 tx=0x7f475c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:53.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.560+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47640bcf70 con 0x7f4770103ff0 2024-09-12T03:03:53.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:53.780+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f477010c300 con 0x7f4770103ff0 2024-09-12T03:03:54.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.781+0000 7f476d600640 1 -- 172.21.15.57:0/4010314773 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f47640bc870 con 0x7f4770103ff0 2024-09-12T03:03:54.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.784+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 msgr2=0x7f47580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.784+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 0x7f47580a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f47700fff40 tx=0x7f475c005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.784+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 msgr2=0x7f477019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.784+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f476402a930 tx=0x7f4764003b20 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 shutdown_connections 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4770109120 0x7f477019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47701049a0 0x7f477019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 --2- 172.21.15.57:0/4010314773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4770103ff0 0x7f477019a670 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 >> 172.21.15.57:0/4010314773 conn(0x7f47700ffa90 msgr2=0x7f4770100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 shutdown_connections 2024-09-12T03:03:54.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:54.785+0000 7f4777600640 1 -- 172.21.15.57:0/4010314773 wait complete. 2024-09-12T03:03:55.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:54 smithi057 ceph-mon[30361]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:55.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:54 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4010314773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:54 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:54 smithi204 ceph-mon[35013]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:54 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/4010314773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:54 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:54 smithi181 ceph-mon[35320]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:03:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:54 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/4010314773' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:54 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T03:03:55.550 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:03:55.941 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:56.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:03:56.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:03:56.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:03:56.080 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:56.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-12T03:03:56.081 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:55 smithi057 ceph-mon[30361]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-12T03:03:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:03:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:03:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:03:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-12T03:03:56.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:55 smithi204 ceph-mon[35013]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-12T03:03:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:55 smithi181 ceph-mon[35320]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 -- 172.21.15.57:0/801678773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08102790 msgr2=0x7f8e081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 --2- 172.21.15.57:0/801678773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08102790 0x7f8e081067e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009900 tx=0x7f8df002a410 comp rx=0 tx=0).stop 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 -- 172.21.15.57:0/801678773 shutdown_connections 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 --2- 172.21.15.57:0/801678773 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08106f10 0x7f8e0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 --2- 172.21.15.57:0/801678773 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08102790 0x7f8e081067e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 --2- 172.21.15.57:0/801678773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08101de0 0x7f8e081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 -- 172.21.15.57:0/801678773 >> 172.21.15.57:0/801678773 conn(0x7f8e080fd8c0 msgr2=0x7f8e080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 -- 172.21.15.57:0/801678773 shutdown_connections 2024-09-12T03:03:56.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.504+0000 7f8e0d200640 1 -- 172.21.15.57:0/801678773 wait complete. 2024-09-12T03:03:56.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 Processor -- start 2024-09-12T03:03:56.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 -- start start 2024-09-12T03:03:56.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 0x7f8e0819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08106f10 0x7f8e0819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08111f70 con 0x7f8e08101de0 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08111df0 con 0x7f8e08102790 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.505+0000 7f8e0d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081120f0 con 0x7f8e08106f10 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8dfd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 0x7f8e0819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59720/0 (socket says 172.21.15.57:59720) 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 -- 172.21.15.57:0/29056928 learned_addr learned my addr 172.21.15.57:0/29056928 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:56.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e06800640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08106f10 0x7f8e0819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 -- 172.21.15.57:0/29056928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08106f10 msgr2=0x7f8e0819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08106f10 0x7f8e0819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 -- 172.21.15.57:0/29056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 msgr2=0x7f8e0819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 0x7f8e0819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8e05e00640 1 -- 172.21.15.57:0/29056928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0819f5f0 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.506+0000 7f8dfd400640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 0x7f8e0819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8e05e00640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9f0 tx=0x7f8df800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df800cd80 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df8005500 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0819f8e0 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8010690 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.507+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081a71c0 con 0x7f8e08101de0 2024-09-12T03:03:56.509 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.508+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e081021c0 con 0x7f8e08101de0 2024-09-12T03:03:56.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.510+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8df80046c0 con 0x7f8e08101de0 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.512+0000 7f8dff400640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 0x7f8dec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.513+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8df80cf780 con 0x7f8e08101de0 2024-09-12T03:03:56.513 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.513+0000 7f8dfd400640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 0x7f8dec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:56.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.514+0000 7f8dfd400640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 0x7f8dec0a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009230 tx=0x7f8df00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:56.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.516+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8df809c320 con 0x7f8e08101de0 2024-09-12T03:03:56.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.737+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f8e081a73f0 con 0x7f8e08101de0 2024-09-12T03:03:56.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.800+0000 7f8dff400640 1 -- 172.21.15.57:0/29056928 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f8df8086020 con 0x7f8e08101de0 2024-09-12T03:03:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 msgr2=0x7f8dec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 0x7f8dec0a50c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009230 tx=0x7f8df00058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 msgr2=0x7f8e0819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9f0 tx=0x7f8df800edf0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 shutdown_connections 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8dec0a2c00 0x7f8dec0a50c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.802+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e08106f10 0x7f8e0819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.803+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e08102790 0x7f8e0819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.803+0000 7f8e0d200640 1 --2- 172.21.15.57:0/29056928 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e08101de0 0x7f8e0819a640 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.803+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 >> 172.21.15.57:0/29056928 conn(0x7f8e080fd8c0 msgr2=0x7f8e080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.803+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 shutdown_connections 2024-09-12T03:03:56.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:56.803+0000 7f8e0d200640 1 -- 172.21.15.57:0/29056928 wait complete. 2024-09-12T03:03:57.015 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:active 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: Cluster is now healthy 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:03:57.016 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:56 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:03:57.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T03:03:57.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:active 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: Cluster is now healthy 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:03:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:56 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] up:active 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: Cluster is now healthy 2024-09-12T03:03:57.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-12T03:03:57.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-12T03:03:57.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-12T03:03:57.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:56 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: 2024-09-12T03:03:57.412+0000 7f61664d6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T03:03:57.703 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: 2024-09-12T03:03:57.412+0000 7f61664d6640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T03:03:57.704 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:03:58.086 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:58.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:57 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:03:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:57 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:active 2024-09-12T03:03:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:57 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:03:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:57 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:57 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:03:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:57 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:active 2024-09-12T03:03:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:57 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:03:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:57 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/29056928' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:03:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] up:active 2024-09-12T03:03:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-12T03:03:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:57 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 -- 172.21.15.57:0/1926435280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041079c0 msgr2=0x7fb604107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 --2- 172.21.15.57:0/1926435280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041079c0 0x7fb604107da0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009930 tx=0x7fb5f402a420 comp rx=0 tx=0).stop 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 -- 172.21.15.57:0/1926435280 shutdown_connections 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 --2- 172.21.15.57:0/1926435280 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb60410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 --2- 172.21.15.57:0/1926435280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041019c0 0x7fb604101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 --2- 172.21.15.57:0/1926435280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041079c0 0x7fb604107da0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 -- 172.21.15.57:0/1926435280 >> 172.21.15.57:0/1926435280 conn(0x7fb6040fd740 msgr2=0x7fb6040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:58.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 -- 172.21.15.57:0/1926435280 shutdown_connections 2024-09-12T03:03:58.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.654+0000 7fb609400640 1 -- 172.21.15.57:0/1926435280 wait complete. 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.655+0000 7fb609400640 1 Processor -- start 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.655+0000 7fb609400640 1 -- start start 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041019c0 0x7fb6041088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041079c0 0x7fb604109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb604111e10 con 0x7fb604102360 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb604111c90 con 0x7fb6041079c0 2024-09-12T03:03:58.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb609400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb604111f90 con 0x7fb6041019c0 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb603e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041019c0 0x7fb6041088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48390/0 (socket says 172.21.15.57:48390) 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 -- 172.21.15.57:0/2016916558 learned_addr learned my addr 172.21.15.57:0/2016916558 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb608a00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041079c0 0x7fb604109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 -- 172.21.15.57:0/2016916558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041019c0 msgr2=0x7fb6041088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041019c0 0x7fb6041088f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 -- 172.21.15.57:0/2016916558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041079c0 msgr2=0x7fb604109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.656+0000 7fb5fbe00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041079c0 0x7fb604109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:58.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb5fbe00640 1 -- 172.21.15.57:0/2016916558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6041aab90 con 0x7fb604102360 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb5fbe00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00ea20 tx=0x7fb5ec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00cd80 con 0x7fb604102360 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec010910 con 0x7fb604102360 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6041aae80 con 0x7fb604102360 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec018920 con 0x7fb604102360 2024-09-12T03:03:58.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.657+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6041ab2e0 con 0x7fb604102360 2024-09-12T03:03:58.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.658+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb604103100 con 0x7fb604102360 2024-09-12T03:03:58.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.660+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb5ec010ac0 con 0x7fb604102360 2024-09-12T03:03:58.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.662+0000 7fb601600640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:58.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.662+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb5ec0cf710 con 0x7fb604102360 2024-09-12T03:03:58.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.662+0000 7fb603e00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:58.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.663+0000 7fb603e00640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 0x7fb5e80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006010 tx=0x7fb5f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:58.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.666+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5ec09bc60 con 0x7fb604102360 2024-09-12T03:03:58.892 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:58.892+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb6041ab660 con 0x7fb604102360 2024-09-12T03:03:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:58 smithi057 ceph-mon[30361]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-12T03:03:59.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:58 smithi204 ceph-mon[35013]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-12T03:03:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:58 smithi181 ceph-mon[35320]: pgmap v333: 129 pgs: 129 active+clean; 584 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-12T03:03:59.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.822+0000 7fb601600640 1 -- 172.21.15.57:0/2016916558 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fb5ec09ff60 con 0x7fb604102360 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 msgr2=0x7fb5e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 0x7fb5e80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006010 tx=0x7fb5f4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 msgr2=0x7fb604108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00ea20 tx=0x7fb5ec00eee0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 shutdown_connections 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb5e80a2b80 0x7fb5e80a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6041079c0 0x7fb604109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb604102360 0x7fb604108e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 --2- 172.21.15.57:0/2016916558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6041019c0 0x7fb6041088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.825+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 >> 172.21.15.57:0/2016916558 conn(0x7fb6040fd740 msgr2=0x7fb6040fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.826+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 shutdown_connections 2024-09-12T03:03:59.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:03:59.826+0000 7fb609400640 1 -- 172.21.15.57:0/2016916558 wait complete. 2024-09-12T03:04:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:03:59 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:04:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:03:59 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:04:00.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:03:59 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:04:00.658 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T03:04:00.658 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:04:00.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:00 smithi057 ceph-mon[30361]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:04:00.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:00 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:04:00.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:00 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:01.043 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:00 smithi204 ceph-mon[35013]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:04:01.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:00 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:04:01.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:00 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:00 smithi181 ceph-mon[35320]: pgmap v334: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:04:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:00 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2016916558' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:04:01.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:00 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 -- 172.21.15.57:0/3629099440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc102b90 msgr2=0x7fc1fc102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 --2- 172.21.15.57:0/3629099440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc102b90 0x7fc1fc102f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009900 tx=0x7fc1ec02a480 comp rx=0 tx=0).stop 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 -- 172.21.15.57:0/3629099440 shutdown_connections 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 --2- 172.21.15.57:0/3629099440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 --2- 172.21.15.57:0/3629099440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc1034b0 0x7fc1fc10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.621 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 --2- 172.21.15.57:0/3629099440 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc102b90 0x7fc1fc102f70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 -- 172.21.15.57:0/3629099440 >> 172.21.15.57:0/3629099440 conn(0x7fc1fc0fc610 msgr2=0x7fc1fc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 -- 172.21.15.57:0/3629099440 shutdown_connections 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.595+0000 7fc1fb200640 1 -- 172.21.15.57:0/3629099440 wait complete. 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.596+0000 7fc1fb200640 1 Processor -- start 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.596+0000 7fc1fb200640 1 -- start start 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc102b90 0x7fc1fc1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1034b0 0x7fc1fc100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc111e10 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc111c90 con 0x7fc1fc102b90 2024-09-12T03:04:01.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc111f90 con 0x7fc1fc1034b0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fa800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48416/0 (socket says 172.21.15.57:48416) 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1fa800640 1 -- 172.21.15.57:0/898882052 learned_addr learned my addr 172.21.15.57:0/898882052 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1f1400640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1034b0 0x7fc1fc100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.597+0000 7fc1f9e00640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc102b90 0x7fc1fc1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 -- 172.21.15.57:0/898882052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1034b0 msgr2=0x7fc1fc100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1034b0 0x7fc1fc100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 -- 172.21.15.57:0/898882052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc102b90 msgr2=0x7fc1fc1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc102b90 0x7fc1fc1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 -- 172.21.15.57:0/898882052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc1a6760 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fa800640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eb10 tx=0x7fc1f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004b20 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1f4004cc0 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4010760 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.598+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc1a6a50 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.599+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1a6f40 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.600+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc102f70 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.602+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1f4005020 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.603+0000 7fc1f3400640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 0x7fc1e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.603+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fc1f40cf9b0 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.603+0000 7fc1f9e00640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 0x7fc1e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.604+0000 7fc1f9e00640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 0x7fc1e00a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec000c00 tx=0x7fc1ec005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:01.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.607+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1f409c550 con 0x7fc1fc10bfc0 2024-09-12T03:04:01.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.815+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc1fc062630 con 0x7fc1e00a2c00 2024-09-12T03:04:01.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.847+0000 7fc1f3400640 1 -- 172.21.15.57:0/898882052 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1fc062630 con 0x7fc1e00a2c00 2024-09-12T03:04:01.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 msgr2=0x7fc1e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:01.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 0x7fc1e00a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec000c00 tx=0x7fc1ec005b50 comp rx=0 tx=0).stop 2024-09-12T03:04:01.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 msgr2=0x7fc1fc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eb10 tx=0x7fc1f400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 shutdown_connections 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fc1e00a2c00 0x7fc1e00a50c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc1fc10bfc0 0x7fc1fc101270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1fc1034b0 0x7fc1fc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 --2- 172.21.15.57:0/898882052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1fc102b90 0x7fc1fc1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.849+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 >> 172.21.15.57:0/898882052 conn(0x7fc1fc0fc610 msgr2=0x7fc1fc0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.850+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 shutdown_connections 2024-09-12T03:04:01.851 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:01.850+0000 7fc1fb200640 1 -- 172.21.15.57:0/898882052 wait complete. 2024-09-12T03:04:02.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:04:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:04:01.822+0000 7f500f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:04:02.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:04:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:04:01.822+0000 7f500f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:04:02.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:04:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:04:01.822+0000 7f500f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:04:02.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:04:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:04:01.822+0000 7f500f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:04:02.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:04:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:04:01.822+0000 7f500f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:04:02.606 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:04:02.985 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:03.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:02 smithi057 ceph-mon[30361]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-12T03:04:03.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:02 smithi057 ceph-mon[30361]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:02 smithi204 ceph-mon[35013]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-12T03:04:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:02 smithi204 ceph-mon[35013]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:02 smithi181 ceph-mon[35320]: pgmap v335: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-09-12T03:04:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:02 smithi181 ceph-mon[35320]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 -- 172.21.15.57:0/2897087379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 msgr2=0x7fe6841089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 --2- 172.21.15.57:0/2897087379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe6841089f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009900 tx=0x7fe66c02a410 comp rx=0 tx=0).stop 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 -- 172.21.15.57:0/2897087379 shutdown_connections 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 --2- 172.21.15.57:0/2897087379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe684110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 --2- 172.21.15.57:0/2897087379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe6841089f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 --2- 172.21.15.57:0/2897087379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 0x7fe6841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 -- 172.21.15.57:0/2897087379 >> 172.21.15.57:0/2897087379 conn(0x7fe6840ffa90 msgr2=0x7fe684101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 -- 172.21.15.57:0/2897087379 shutdown_connections 2024-09-12T03:04:03.616 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.615+0000 7fe689c00640 1 -- 172.21.15.57:0/2897087379 wait complete. 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.616+0000 7fe689c00640 1 Processor -- start 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.616+0000 7fe689c00640 1 -- start start 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.616+0000 7fe689c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 0x7fe68419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.616+0000 7fe689c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.616+0000 7fe689c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe68419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe689c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe684114120 con 0x7fe6841049a0 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe689c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe684113fa0 con 0x7fe684103ff0 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe689c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841142a0 con 0x7fe684109120 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe67a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe68419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:03.618 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe67a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48452/0 (socket says 172.21.15.57:48452) 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe67a000640 1 -- 172.21.15.57:0/2511790031 learned_addr learned my addr 172.21.15.57:0/2511790031 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe682a00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 0x7fe68419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 -- 172.21.15.57:0/2511790031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 msgr2=0x7fe68419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 0x7fe68419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 -- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 msgr2=0x7fe68419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe683400640 1 -- 172.21.15.57:0/2511790031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68419f650 con 0x7fe684109120 2024-09-12T03:04:03.619 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.617+0000 7fe67a000640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe683400640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe68419ef70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00ce20 con 0x7fe684109120 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68419f940 con 0x7fe684109120 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c005020 con 0x7fe684109120 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c010720 con 0x7fe684109120 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.618+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6841a7220 con 0x7fe684109120 2024-09-12T03:04:03.620 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.619+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6841043d0 con 0x7fe684109120 2024-09-12T03:04:03.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.621+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe67c005540 con 0x7fe684109120 2024-09-12T03:04:03.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.623+0000 7fe67be00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 0x7fe6680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:03.623 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.623+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe67c0cfb90 con 0x7fe684109120 2024-09-12T03:04:03.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.623+0000 7fe682a00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 0x7fe6680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:03.624 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.623+0000 7fe682a00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 0x7fe6680a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe674005d00 tx=0x7fe67400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:03.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.625+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe67c09c730 con 0x7fe684109120 2024-09-12T03:04:03.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.844+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe684062630 con 0x7fe6680a2b80 2024-09-12T03:04:03.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.848+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe67c0986a0 con 0x7fe684109120 2024-09-12T03:04:03.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.871+0000 7fe67be00640 1 -- 172.21.15.57:0/2511790031 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe684062630 con 0x7fe6680a2b80 2024-09-12T03:04:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 msgr2=0x7fe6680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 0x7fe6680a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe674005d00 tx=0x7fe67400b410 comp rx=0 tx=0).stop 2024-09-12T03:04:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 msgr2=0x7fe68419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:03.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe68419ef70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c00ef10 comp rx=0 tx=0).stop 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 shutdown_connections 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe6680a2b80 0x7fe6680a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe684109120 0x7fe68419ef70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe6841049a0 0x7fe68419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 --2- 172.21.15.57:0/2511790031 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe684103ff0 0x7fe68419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.873+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 >> 172.21.15.57:0/2511790031 conn(0x7fe6840ffa90 msgr2=0x7fe684100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.874+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 shutdown_connections 2024-09-12T03:04:03.875 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:03.874+0000 7fe689c00640 1 -- 172.21.15.57:0/2511790031 wait complete. 2024-09-12T03:04:04.607 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:04:04.982 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:05.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:04 smithi057 ceph-mon[30361]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T03:04:05.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:04 smithi057 ceph-mon[30361]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:05.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:04 smithi057 ceph-mon[30361]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:04:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:04 smithi204 ceph-mon[35013]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T03:04:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:04 smithi204 ceph-mon[35013]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:04 smithi204 ceph-mon[35013]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:04:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:04 smithi181 ceph-mon[35320]: pgmap v336: 129 pgs: 129 active+clean; 584 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-12T03:04:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:04 smithi181 ceph-mon[35320]: from='client.24736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:04 smithi181 ceph-mon[35320]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-12T03:04:05.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 -- 172.21.15.57:0/2480525596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204101df0 msgr2=0x7fb2041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:05.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 --2- 172.21.15.57:0/2480525596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204101df0 0x7fb2041021d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009900 tx=0x7fb1f402a400 comp rx=0 tx=0).stop 2024-09-12T03:04:05.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 -- 172.21.15.57:0/2480525596 shutdown_connections 2024-09-12T03:04:05.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 --2- 172.21.15.57:0/2480525596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204106f20 0x7fb20410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 --2- 172.21.15.57:0/2480525596 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 0x7fb2041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 --2- 172.21.15.57:0/2480525596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204101df0 0x7fb2041021d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 -- 172.21.15.57:0/2480525596 >> 172.21.15.57:0/2480525596 conn(0x7fb2040fd8b0 msgr2=0x7fb2040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 -- 172.21.15.57:0/2480525596 shutdown_connections 2024-09-12T03:04:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.591+0000 7fb20a200640 1 -- 172.21.15.57:0/2480525596 wait complete. 2024-09-12T03:04:05.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.592+0000 7fb20a200640 1 Processor -- start 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.592+0000 7fb20a200640 1 -- start start 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 0x7fb204074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 0x7fb204074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb204111f80 con 0x7fb2041027a0 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb204111e00 con 0x7fb204106f20 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb20a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb204112100 con 0x7fb204101df0 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb202a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 0x7fb204074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:41906/0 (socket says 172.21.15.57:41906) 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203400640 1 -- 172.21.15.57:0/1909906607 learned_addr learned my addr 172.21.15.57:0/1909906607 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:05.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203e00640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 0x7fb204074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203400640 1 -- 172.21.15.57:0/1909906607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 msgr2=0x7fb204074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:05.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.593+0000 7fb203400640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 0x7fb204074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb203400640 1 -- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 msgr2=0x7fb204074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb203400640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 0x7fb204074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb203400640 1 -- 172.21.15.57:0/1909906607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2041a6920 con 0x7fb204101df0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb203e00640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 0x7fb204074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb203400640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006010 tx=0x7fb1f4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f403f030 con 0x7fb204101df0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f4004440 con 0x7fb204101df0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2041a6bb0 con 0x7fb204101df0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.594+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4035070 con 0x7fb204101df0 2024-09-12T03:04:05.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.595+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2041a70e0 con 0x7fb204101df0 2024-09-12T03:04:05.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.596+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2041021d0 con 0x7fb204101df0 2024-09-12T03:04:05.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.597+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb1f4042030 con 0x7fb204101df0 2024-09-12T03:04:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.599+0000 7fb1fbe00640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 0x7fb1e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:05.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.599+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb1f40ef850 con 0x7fb204101df0 2024-09-12T03:04:05.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.599+0000 7fb202a00640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 0x7fb1e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:05.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.600+0000 7fb202a00640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 0x7fb1e80a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00095f0 tx=0x7fb1f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:05.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.603+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1f40bbd00 con 0x7fb204101df0 2024-09-12T03:04:05.823 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.822+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fb20410a170 con 0x7fb1e80a2c90 2024-09-12T03:04:05.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.829+0000 7fb1fbe00640 1 -- 172.21.15.57:0/1909906607 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb1f000e4a0 con 0x7fb1e80a2c90 2024-09-12T03:04:05.830 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/qa/sv_0/d7b27877-5120-49e2-ac34-60a607f02611 2024-09-12T03:04:05.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 msgr2=0x7fb1e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:05.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 0x7fb1e80a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00095f0 tx=0x7fb1f0005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:05.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 msgr2=0x7fb2040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:05.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4006010 tx=0x7fb1f4004040 comp rx=0 tx=0).stop 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 shutdown_connections 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb1e80a2c90 0x7fb1e80a5150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb204106f20 0x7fb204074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb2041027a0 0x7fb204074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 --2- 172.21.15.57:0/1909906607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb204101df0 0x7fb2040744f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 >> 172.21.15.57:0/1909906607 conn(0x7fb2040fd8b0 msgr2=0x7fb2040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 shutdown_connections 2024-09-12T03:04:05.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:05.832+0000 7fb20a200640 1 -- 172.21.15.57:0/1909906607 wait complete. 2024-09-12T03:04:06.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:05 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:06.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:05 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:06.180 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:05 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:06.615 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:04:07.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:06 smithi057 ceph-mon[30361]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T03:04:07.105 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:06 smithi057 ceph-mon[30361]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:07.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:06 smithi204 ceph-mon[35013]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T03:04:07.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:06 smithi204 ceph-mon[35013]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:07.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:06 smithi181 ceph-mon[35320]: pgmap v337: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-12T03:04:07.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:06 smithi181 ceph-mon[35320]: from='client.24742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:07.495 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:07.962 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:07.962 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:07.962 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:07.962 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:07 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 -- 172.21.15.57:0/3533027531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee8109120 msgr2=0x7fdee8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 --2- 172.21.15.57:0/3533027531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee8109120 0x7fdee8110ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009ca0 tx=0x7fdee002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 -- 172.21.15.57:0/3533027531 shutdown_connections 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 --2- 172.21.15.57:0/3533027531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee8109120 0x7fdee8110ca0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 --2- 172.21.15.57:0/3533027531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee81049a0 0x7fdee81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 --2- 172.21.15.57:0/3533027531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8103ff0 0x7fdee81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.079 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 -- 172.21.15.57:0/3533027531 >> 172.21.15.57:0/3533027531 conn(0x7fdee80ffa90 msgr2=0x7fdee8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 -- 172.21.15.57:0/3533027531 shutdown_connections 2024-09-12T03:04:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.078+0000 7fdeef600640 1 -- 172.21.15.57:0/3533027531 wait complete. 2024-09-12T03:04:08.080 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.079+0000 7fdeef600640 1 Processor -- start 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.079+0000 7fdeef600640 1 -- start start 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 0x7fdee819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 0x7fdee819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdee81141a0 con 0x7fdee81049a0 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8114020 con 0x7fdee8109120 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeef600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8114320 con 0x7fdee8103ff0 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35892/0 (socket says 172.21.15.57:35892) 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdedfe00640 1 -- 172.21.15.57:0/1863438676 learned_addr learned my addr 172.21.15.57:0/1863438676 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeed600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 0x7fdee819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.081 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.080+0000 7fdeecc00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 0x7fdee819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 -- 172.21.15.57:0/1863438676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 msgr2=0x7fdee819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 0x7fdee819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.082 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 -- 172.21.15.57:0/1863438676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 msgr2=0x7fdee819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 0x7fdee819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 -- 172.21.15.57:0/1863438676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee819f590 con 0x7fdee81049a0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdeecc00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 0x7fdee819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdeed600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 0x7fdee819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdedfe00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fded000e9f0 tx=0x7fded000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded000cd80 con 0x7fdee81049a0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee819f880 con 0x7fdee81049a0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded0005500 con 0x7fdee81049a0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0010690 con 0x7fdee81049a0 2024-09-12T03:04:08.083 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.081+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee81a7160 con 0x7fdee81049a0 2024-09-12T03:04:08.084 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.082+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee81043d0 con 0x7fdee81049a0 2024-09-12T03:04:08.085 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.084+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fded00046c0 con 0x7fdee81049a0 2024-09-12T03:04:08.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.085+0000 7fdee6000640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 0x7fdecc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:08.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.086+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fded00cfa90 con 0x7fdee81049a0 2024-09-12T03:04:08.086 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.086+0000 7fdeecc00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 0x7fdecc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:08.087 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.087+0000 7fdeecc00640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 0x7fdecc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fded8005fd0 tx=0x7fded800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:08.090 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.090+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fded009c4d0 con 0x7fdee81049a0 2024-09-12T03:04:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:07 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:07 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:08.307 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.306+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdee8100cc0 con 0x7fdecc0a2c10 2024-09-12T03:04:08.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.334+0000 7fdee6000640 1 -- 172.21.15.57:0/1863438676 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdee8100cc0 con 0x7fdecc0a2c10 2024-09-12T03:04:08.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 msgr2=0x7fdecc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 0x7fdecc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fded8005fd0 tx=0x7fded800b5f0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 msgr2=0x7fdee819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:08.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fded000e9f0 tx=0x7fded000edf0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 shutdown_connections 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdecc0a2c10 0x7fdecc0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee8109120 0x7fdee819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee81049a0 0x7fdee819ab20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 --2- 172.21.15.57:0/1863438676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee8103ff0 0x7fdee819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.336+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 >> 172.21.15.57:0/1863438676 conn(0x7fdee80ffa90 msgr2=0x7fdee81005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.337+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 shutdown_connections 2024-09-12T03:04:08.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:08.337+0000 7fdeef600640 1 -- 172.21.15.57:0/1863438676 wait complete. 2024-09-12T03:04:09.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:08 smithi057 ceph-mon[30361]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:04:09.117 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:08 smithi057 ceph-mon[30361]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:09.155 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:04:09.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:08 smithi181 ceph-mon[35320]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:04:09.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:08 smithi181 ceph-mon[35320]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:09.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:08 smithi204 ceph-mon[35013]: pgmap v338: 129 pgs: 129 active+clean; 604 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-12T03:04:09.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:08 smithi204 ceph-mon[35013]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:09.571 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 -- 172.21.15.57:0/3024133887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404103ff0 msgr2=0x7f54041043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 --2- 172.21.15.57:0/3024133887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404103ff0 0x7f54041043d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 -- 172.21.15.57:0/3024133887 shutdown_connections 2024-09-12T03:04:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 --2- 172.21.15.57:0/3024133887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5404109120 0x7f5404110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 --2- 172.21.15.57:0/3024133887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54041049a0 0x7f54041089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 --2- 172.21.15.57:0/3024133887 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404103ff0 0x7f54041043d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 -- 172.21.15.57:0/3024133887 >> 172.21.15.57:0/3024133887 conn(0x7f54040ffa90 msgr2=0x7f5404101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 -- 172.21.15.57:0/3024133887 shutdown_connections 2024-09-12T03:04:10.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.168+0000 7f5408c00640 1 -- 172.21.15.57:0/3024133887 wait complete. 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.169+0000 7f5408c00640 1 Processor -- start 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.169+0000 7f5408c00640 1 -- start start 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5404103ff0 0x7f540419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 0x7f540419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5404114120 con 0x7f5404109120 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5404113fa0 con 0x7f5404103ff0 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5408c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f54041142a0 con 0x7f54041049a0 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5402800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5402800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35920/0 (socket says 172.21.15.57:35920) 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5402800640 1 -- 172.21.15.57:0/1181130114 learned_addr learned my addr 172.21.15.57:0/1181130114 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:10.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5401400640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 0x7f540419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.170+0000 7f5401e00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5404103ff0 0x7f540419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 -- 172.21.15.57:0/1181130114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 msgr2=0x7f540419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 0x7f540419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 -- 172.21.15.57:0/1181130114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5404103ff0 msgr2=0x7f540419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5404103ff0 0x7f540419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 -- 172.21.15.57:0/1181130114 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540419f590 con 0x7f5404109120 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5401400640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 0x7f540419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5402800640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00eb10 tx=0x7f53fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc004b20 con 0x7f5404109120 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53fc004cc0 con 0x7f5404109120 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540419f880 con 0x7f5404109120 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.171+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc010760 con 0x7f5404109120 2024-09-12T03:04:10.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.172+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54041a7160 con 0x7f5404109120 2024-09-12T03:04:10.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.173+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54041043d0 con 0x7f5404109120 2024-09-12T03:04:10.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.175+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f53fc005020 con 0x7f5404109120 2024-09-12T03:04:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.176+0000 7f53faa00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 0x7f53e80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:10.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.176+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f53fc0cfa20 con 0x7f5404109120 2024-09-12T03:04:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.176+0000 7f5401e00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 0x7f53e80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:10.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.177+0000 7f5401e00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 0x7f53e80a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009230 tx=0x7f53f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:10.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.180+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53fc09c460 con 0x7f5404109120 2024-09-12T03:04:10.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.405+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f5404062630 con 0x7f53e80a2c90 2024-09-12T03:04:10.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.410+0000 7f53faa00640 1 -- 172.21.15.57:0/1181130114 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5404062630 con 0x7f53e80a2c90 2024-09-12T03:04:10.410 INFO:teuthology.orchestra.run.smithi057.stdout:/volumes/qa/sv_1/7a03ebfb-226f-403a-b746-3dd303bbfeee 2024-09-12T03:04:10.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 msgr2=0x7f53e80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:10.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 0x7f53e80a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009230 tx=0x7f53f40058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 msgr2=0x7f540419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:10.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00eb10 tx=0x7f53fc00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 shutdown_connections 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f53e80a2c90 0x7f53e80a5150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5404109120 0x7f540419eeb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54041049a0 0x7f540419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 --2- 172.21.15.57:0/1181130114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5404103ff0 0x7f540419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.412+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 >> 172.21.15.57:0/1181130114 conn(0x7f54040ffa90 msgr2=0x7f5404100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.413+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 shutdown_connections 2024-09-12T03:04:10.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:10.413+0000 7f5408c00640 1 -- 172.21.15.57:0/1181130114 wait complete. 2024-09-12T03:04:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:10.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:10.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:04:11.190 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:11.575 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:11.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:11 smithi057 ceph-mon[30361]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T03:04:11.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:11 smithi057 ceph-mon[30361]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:11 smithi204 ceph-mon[35013]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T03:04:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:11 smithi204 ceph-mon[35013]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:11 smithi181 ceph-mon[35320]: pgmap v339: 129 pgs: 129 active+clean; 605 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-09-12T03:04:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:11 smithi181 ceph-mon[35320]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:04:12.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 -- 172.21.15.57:0/2775034802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44076720 msgr2=0x7f7a44076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2775034802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44076720 0x7f7a44076b00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009900 tx=0x7f7a3802a410 comp rx=0 tx=0).stop 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 -- 172.21.15.57:0/2775034802 shutdown_connections 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2775034802 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44112870 0x7f7a44114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2775034802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 0x7f7a44112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2775034802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44076720 0x7f7a44076b00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 -- 172.21.15.57:0/2775034802 >> 172.21.15.57:0/2775034802 conn(0x7f7a440ff8d0 msgr2=0x7f7a44101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 -- 172.21.15.57:0/2775034802 shutdown_connections 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.162+0000 7f7a4b000640 1 -- 172.21.15.57:0/2775034802 wait complete. 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.163+0000 7f7a4b000640 1 Processor -- start 2024-09-12T03:04:12.166 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.163+0000 7f7a4b000640 1 -- start start 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 0x7f7a441a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44112870 0x7f7a4419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7a441183e0 con 0x7f7a44076720 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44118260 con 0x7f7a44112870 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a44118560 con 0x7f7a44077040 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a49c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a49c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35946/0 (socket says 172.21.15.57:35946) 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a49c00640 1 -- 172.21.15.57:0/2483413642 learned_addr learned my addr 172.21.15.57:0/2483413642 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a4a600640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44112870 0x7f7a4419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.164+0000 7f7a49200640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 0x7f7a441a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 -- 172.21.15.57:0/2483413642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 msgr2=0x7f7a441a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:12.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 0x7f7a441a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 -- 172.21.15.57:0/2483413642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44112870 msgr2=0x7f7a4419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44112870 0x7f7a4419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 -- 172.21.15.57:0/2483413642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4419ef60 con 0x7f7a44076720 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a49c00640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006010 tx=0x7f7a38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3803f030 con 0x7f7a44076720 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a38004440 con 0x7f7a44076720 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38035070 con 0x7f7a44076720 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4419f1f0 con 0x7f7a44076720 2024-09-12T03:04:12.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.165+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a441ab3d0 con 0x7f7a44076720 2024-09-12T03:04:12.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.166+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a44077e60 con 0x7f7a44076720 2024-09-12T03:04:12.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.168+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a38042030 con 0x7f7a44076720 2024-09-12T03:04:12.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.170+0000 7f7a42000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 0x7f7a2c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:12.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.170+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7a380ef860 con 0x7f7a44076720 2024-09-12T03:04:12.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.170+0000 7f7a49200640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 0x7f7a2c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:12.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.171+0000 7f7a49200640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 0x7f7a2c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a4419ff40 tx=0x7f7a30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:12.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.174+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a380bbc50 con 0x7f7a44076720 2024-09-12T03:04:12.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.402+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a44076b00 con 0x7f7a44076720 2024-09-12T03:04:12.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.403+0000 7f7a42000640 1 -- 172.21.15.57:0/2483413642 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7f7a380bff50 con 0x7f7a44076720 2024-09-12T03:04:12.402 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:12.403 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:04:12.403 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:04:12.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 msgr2=0x7f7a2c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:12.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 0x7f7a2c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a4419ff40 tx=0x7f7a30005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:12.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 msgr2=0x7f7a441a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:12.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006010 tx=0x7f7a38004040 comp rx=0 tx=0).stop 2024-09-12T03:04:12.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 shutdown_connections 2024-09-12T03:04:12.405 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f7a2c0a2c90 0x7f7a2c0a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a44112870 0x7f7a4419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a44077040 0x7f7a441a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 --2- 172.21.15.57:0/2483413642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7a44076720 0x7f7a441a4620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.405+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 >> 172.21.15.57:0/2483413642 conn(0x7f7a440ff8d0 msgr2=0x7f7a44101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.406+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 shutdown_connections 2024-09-12T03:04:12.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:12.406+0000 7f7a4b000640 1 -- 172.21.15.57:0/2483413642 wait complete. 2024-09-12T03:04:13.150 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:04:13.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:13 smithi057 ceph-mon[30361]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T03:04:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:13 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2483413642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:13.531 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:13 smithi204 ceph-mon[35013]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T03:04:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:13 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2483413642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:13 smithi181 ceph-mon[35320]: pgmap v340: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-12T03:04:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:13 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2483413642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 -- 172.21.15.57:0/3355576628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341019c0 msgr2=0x7f9234101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 --2- 172.21.15.57:0/3355576628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341019c0 0x7f9234101e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f921c009930 tx=0x7f921c02a420 comp rx=0 tx=0).stop 2024-09-12T03:04:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 -- 172.21.15.57:0/3355576628 shutdown_connections 2024-09-12T03:04:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 --2- 172.21.15.57:0/3355576628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f923410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 --2- 172.21.15.57:0/3355576628 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341019c0 0x7f9234101e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 --2- 172.21.15.57:0/3355576628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341079c0 0x7f9234107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 -- 172.21.15.57:0/3355576628 >> 172.21.15.57:0/3355576628 conn(0x7f92340fd740 msgr2=0x7f92340ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 -- 172.21.15.57:0/3355576628 shutdown_connections 2024-09-12T03:04:14.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.205+0000 7f923a000640 1 -- 172.21.15.57:0/3355576628 wait complete. 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.206+0000 7f923a000640 1 Processor -- start 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.206+0000 7f923a000640 1 -- start start 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341019c0 0x7f92340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9234111dc0 con 0x7f92341079c0 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9234111c40 con 0x7f9234102360 2024-09-12T03:04:14.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f923a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9234111f40 con 0x7f92341019c0 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f922be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f9239600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f9238c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341019c0 0x7f92340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f9239600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35968/0 (socket says 172.21.15.57:35968) 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f922be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58340/0 (socket says 172.21.15.57:58340) 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.207+0000 7f9239600640 1 -- 172.21.15.57:0/3996093205 learned_addr learned my addr 172.21.15.57:0/3996093205 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 -- 172.21.15.57:0/3996093205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341019c0 msgr2=0x7f92340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341019c0 0x7f92340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 -- 172.21.15.57:0/3996093205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 msgr2=0x7f9234074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 -- 172.21.15.57:0/3996093205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92341a6800 con 0x7f92341079c0 2024-09-12T03:04:14.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f922be00640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:14.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9239600640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f922c00eaa0 tx=0x7f922c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:14.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c00cdb0 con 0x7f92341079c0 2024-09-12T03:04:14.209 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922c005020 con 0x7f92341079c0 2024-09-12T03:04:14.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92341a6af0 con 0x7f92341079c0 2024-09-12T03:04:14.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c0116b0 con 0x7f92341079c0 2024-09-12T03:04:14.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.208+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92341a6fb0 con 0x7f92341079c0 2024-09-12T03:04:14.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.210+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9234101e20 con 0x7f92341079c0 2024-09-12T03:04:14.210 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.211+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f922c005560 con 0x7f92341079c0 2024-09-12T03:04:14.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.212+0000 7f9232000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 0x7f92180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:14.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.213+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f922c0cf760 con 0x7f92341079c0 2024-09-12T03:04:14.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.213+0000 7f9238c00640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 0x7f92180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:14.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.213+0000 7f9238c00640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 0x7f92180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9224009b50 tx=0x7f9224005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:14.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.217+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f922c016030 con 0x7f92341079c0 2024-09-12T03:04:14.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.403+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9234109f30 con 0x7f92341079c0 2024-09-12T03:04:14.404 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.404+0000 7f9232000640 1 -- 172.21.15.57:0/3996093205 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f922c09bb50 con 0x7f92341079c0 2024-09-12T03:04:14.404 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:14.405 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:41.651986+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:04:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 msgr2=0x7f92180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 0x7f92180a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9224009b50 tx=0x7f9224005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 msgr2=0x7f9234074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f922c00eaa0 tx=0x7f922c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:04:14.406 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 shutdown_connections 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f92180a2c10 0x7f92180a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92341079c0 0x7f9234074f70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9234102360 0x7f9234074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 --2- 172.21.15.57:0/3996093205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92341019c0 0x7f92340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 >> 172.21.15.57:0/3996093205 conn(0x7f92340fd740 msgr2=0x7f92340fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 shutdown_connections 2024-09-12T03:04:14.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:14.407+0000 7f923a000640 1 -- 172.21.15.57:0/3996093205 wait complete. 2024-09-12T03:04:15.196 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:04:15.196 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:04:15.574 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:15.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:15 smithi057 ceph-mon[30361]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:04:15.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:15 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3996093205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:15 smithi204 ceph-mon[35013]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:04:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:15 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3996093205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:15 smithi181 ceph-mon[35320]: pgmap v341: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:04:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:15 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3996093205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:16.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 -- 172.21.15.57:0/487853892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 msgr2=0x7f6ed8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 --2- 172.21.15.57:0/487853892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed8108830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec402a410 comp rx=0 tx=0).stop 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 -- 172.21.15.57:0/487853892 shutdown_connections 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 --2- 172.21.15.57:0/487853892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 0x7f6ed8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 --2- 172.21.15.57:0/487853892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed8108830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 --2- 172.21.15.57:0/487853892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 -- 172.21.15.57:0/487853892 >> 172.21.15.57:0/487853892 conn(0x7f6ed80ff910 msgr2=0x7f6ed8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 -- 172.21.15.57:0/487853892 shutdown_connections 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.165+0000 7f6edd800640 1 -- 172.21.15.57:0/487853892 wait complete. 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.166+0000 7f6edd800640 1 Processor -- start 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.166+0000 7f6edd800640 1 -- start start 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.166+0000 7f6edd800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:16.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.166+0000 7f6edd800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.166+0000 7f6edd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6edd800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113f80 con 0x7f6ed81047e0 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6edd800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113e00 con 0x7f6ed8103e30 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6edd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114100 con 0x7f6ed8108f60 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:35986/0 (socket says 172.21.15.57:35986) 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7400640 1 -- 172.21.15.57:0/3635062161 learned_addr learned my addr 172.21.15.57:0/3635062161 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6edce00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 -- 172.21.15.57:0/3635062161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 msgr2=0x7f6ed819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 -- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 msgr2=0x7f6ed819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7e00640 1 -- 172.21.15.57:0/3635062161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed819f410 con 0x7f6ed8103e30 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.167+0000 7f6ed7400640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6ed7e00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800e9f0 tx=0x7f6ec800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800ccc0 con 0x7f6ed8103e30 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed819f700 con 0x7f6ed8103e30 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec8010910 con 0x7f6ed8103e30 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8018960 con 0x7f6ed8103e30 2024-09-12T03:04:16.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.168+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a6fe0 con 0x7f6ed8103e30 2024-09-12T03:04:16.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.170+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8105310 con 0x7f6ed8103e30 2024-09-12T03:04:16.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.171+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6ec8010ac0 con 0x7f6ed8103e30 2024-09-12T03:04:16.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.173+0000 7f6ed4c00640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:16.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.173+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6ec80cf920 con 0x7f6ed8103e30 2024-09-12T03:04:16.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.173+0000 7f6ed7400640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:16.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.174+0000 7f6ed7400640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4031550 tx=0x7f6ec4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:16.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.177+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ec809bd10 con 0x7f6ed8103e30 2024-09-12T03:04:16.359 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.359+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ed810c1b0 con 0x7f6ebc0a2c10 2024-09-12T03:04:16.370 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.369+0000 7f6ed4c00640 1 -- 172.21.15.57:0/3635062161 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7f6ec4031780 con 0x7f6ebc0a2c10 2024-09-12T03:04:16.374 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:16.376 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 msgr2=0x7f6ebc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4031550 tx=0x7f6ec4005a20 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 msgr2=0x7f6ed819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800e9f0 tx=0x7f6ec800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 shutdown_connections 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6ebc0a2c10 0x7f6ebc0a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ed8108f60 0x7f6ed819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ed81047e0 0x7f6ed819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 --2- 172.21.15.57:0/3635062161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8103e30 0x7f6ed819a460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:16.379 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 >> 172.21.15.57:0/3635062161 conn(0x7f6ed80ff910 msgr2=0x7f6ed8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 shutdown_connections 2024-09-12T03:04:16.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:16.374+0000 7f6edd800640 1 -- 172.21.15.57:0/3635062161 wait complete. 2024-09-12T03:04:17.197 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-12T03:04:15.727139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651863,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":265,"num_read_kb":250,"num_write":343,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372204,"kb_used_data":14636,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569620,"statfs":{"total":1151940427776,"available":1151559290880,"internally_reserved":0,"allocated":14987264,"data_stored":8343736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54041,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":95,"num_write":33,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003637"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809294+0000","last_change":"2024-09-12T03:03:04.425029+0000","last_active":"2024-09-12T03:03:55.809294+0000","last_peered":"2024-09-12T03:03:55.809294+0000","last_clean":"2024-09-12T03:03:55.809294+0000","last_became_active":"2024-09-12T03:03:04.423326+0000","last_became_peered":"2024-09-12T03:03:04.423326+0000","last_unstale":"2024-09-12T03:03:55.809294+0000","last_undegraded":"2024-09-12T03:03:55.809294+0000","last_fullsized":"2024-09-12T03:03:55.809294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:28.269107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672403+0000","last_change":"2024-09-12T03:03:07.449348+0000","last_active":"2024-09-12T03:03:41.672403+0000","last_peered":"2024-09-12T03:03:41.672403+0000","last_clean":"2024-09-12T03:03:41.672403+0000","last_became_active":"2024-09-12T03:03:07.448813+0000","last_became_peered":"2024-09-12T03:03:07.448813+0000","last_unstale":"2024-09-12T03:03:41.672403+0000","last_undegraded":"2024-09-12T03:03:41.672403+0000","last_fullsized":"2024-09-12T03:03:41.672403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:31.404583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.666957+0000","last_change":"2024-09-12T03:03:17.530646+0000","last_active":"2024-09-12T03:03:41.666957+0000","last_peered":"2024-09-12T03:03:41.666957+0000","last_clean":"2024-09-12T03:03:41.666957+0000","last_became_active":"2024-09-12T03:03:17.530228+0000","last_became_peered":"2024-09-12T03:03:17.530228+0000","last_unstale":"2024-09-12T03:03:41.666957+0000","last_undegraded":"2024-09-12T03:03:41.666957+0000","last_fullsized":"2024-09-12T03:03:41.666957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.763496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672344+0000","last_change":"2024-09-12T03:03:04.415539+0000","last_active":"2024-09-12T03:03:41.672344+0000","last_peered":"2024-09-12T03:03:41.672344+0000","last_clean":"2024-09-12T03:03:41.672344+0000","last_became_active":"2024-09-12T03:03:04.415163+0000","last_became_peered":"2024-09-12T03:03:04.415163+0000","last_unstale":"2024-09-12T03:03:41.672344+0000","last_undegraded":"2024-09-12T03:03:41.672344+0000","last_fullsized":"2024-09-12T03:03:41.672344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:12.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670669+0000","last_change":"2024-09-12T03:03:07.447808+0000","last_active":"2024-09-12T03:03:41.670669+0000","last_peered":"2024-09-12T03:03:41.670669+0000","last_clean":"2024-09-12T03:03:41.670669+0000","last_became_active":"2024-09-12T03:03:07.446760+0000","last_became_peered":"2024-09-12T03:03:07.446760+0000","last_unstale":"2024-09-12T03:03:41.670669+0000","last_undegraded":"2024-09-12T03:03:41.670669+0000","last_fullsized":"2024-09-12T03:03:41.670669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:56.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671481+0000","last_change":"2024-09-12T03:03:17.530408+0000","last_active":"2024-09-12T03:03:41.671481+0000","last_peered":"2024-09-12T03:03:41.671481+0000","last_clean":"2024-09-12T03:03:41.671481+0000","last_became_active":"2024-09-12T03:03:17.530073+0000","last_became_peered":"2024-09-12T03:03:17.530073+0000","last_unstale":"2024-09-12T03:03:41.671481+0000","last_undegraded":"2024-09-12T03:03:41.671481+0000","last_fullsized":"2024-09-12T03:03:41.671481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:57.199786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807592+0000","last_change":"2024-09-12T03:03:04.422168+0000","last_active":"2024-09-12T03:03:55.807592+0000","last_peered":"2024-09-12T03:03:55.807592+0000","last_clean":"2024-09-12T03:03:55.807592+0000","last_became_active":"2024-09-12T03:03:04.421739+0000","last_became_peered":"2024-09-12T03:03:04.421739+0000","last_unstale":"2024-09-12T03:03:55.807592+0000","last_undegraded":"2024-09-12T03:03:55.807592+0000","last_fullsized":"2024-09-12T03:03:55.807592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:28.080796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672110+0000","last_change":"2024-09-12T03:03:07.445256+0000","last_active":"2024-09-12T03:03:41.672110+0000","last_peered":"2024-09-12T03:03:41.672110+0000","last_clean":"2024-09-12T03:03:41.672110+0000","last_became_active":"2024-09-12T03:03:07.444919+0000","last_became_peered":"2024-09-12T03:03:07.444919+0000","last_unstale":"2024-09-12T03:03:41.672110+0000","last_undegraded":"2024-09-12T03:03:41.672110+0000","last_fullsized":"2024-09-12T03:03:41.672110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:42.826440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456763+0000","last_change":"2024-09-12T03:03:42.456763+0000","last_active":"2024-09-12T03:03:42.456763+0000","last_peered":"2024-09-12T03:03:42.456763+0000","last_clean":"2024-09-12T03:03:42.456763+0000","last_became_active":"2024-09-12T03:03:42.456267+0000","last_became_peered":"2024-09-12T03:03:42.456267+0000","last_unstale":"2024-09-12T03:03:42.456763+0000","last_undegraded":"2024-09-12T03:03:42.456763+0000","last_fullsized":"2024-09-12T03:03:42.456763+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:17.092878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672323+0000","last_change":"2024-09-12T03:03:04.423817+0000","last_active":"2024-09-12T03:03:41.672323+0000","last_peered":"2024-09-12T03:03:41.672323+0000","last_clean":"2024-09-12T03:03:41.672323+0000","last_became_active":"2024-09-12T03:03:04.422617+0000","last_became_peered":"2024-09-12T03:03:04.422617+0000","last_unstale":"2024-09-12T03:03:41.672323+0000","last_undegraded":"2024-09-12T03:03:41.672323+0000","last_fullsized":"2024-09-12T03:03:41.672323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.309338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454778+0000","last_change":"2024-09-12T03:03:07.454768+0000","last_active":"2024-09-12T03:03:42.454778+0000","last_peered":"2024-09-12T03:03:42.454778+0000","last_clean":"2024-09-12T03:03:42.454778+0000","last_became_active":"2024-09-12T03:03:07.454414+0000","last_became_peered":"2024-09-12T03:03:07.454414+0000","last_unstale":"2024-09-12T03:03:42.454778+0000","last_undegraded":"2024-09-12T03:03:42.454778+0000","last_fullsized":"2024-09-12T03:03:42.454778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:39.968938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671409+0000","last_change":"2024-09-12T03:03:17.524984+0000","last_active":"2024-09-12T03:03:41.671409+0000","last_peered":"2024-09-12T03:03:41.671409+0000","last_clean":"2024-09-12T03:03:41.671409+0000","last_became_active":"2024-09-12T03:03:17.524516+0000","last_became_peered":"2024-09-12T03:03:17.524516+0000","last_unstale":"2024-09-12T03:03:41.671409+0000","last_undegraded":"2024-09-12T03:03:41.671409+0000","last_fullsized":"2024-09-12T03:03:41.671409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:34.342631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671167+0000","last_change":"2024-09-12T03:03:07.450348+0000","last_active":"2024-09-12T03:03:41.671167+0000","last_peered":"2024-09-12T03:03:41.671167+0000","last_clean":"2024-09-12T03:03:41.671167+0000","last_became_active":"2024-09-12T03:03:07.448968+0000","last_became_peered":"2024-09-12T03:03:07.448968+0000","last_unstale":"2024-09-12T03:03:41.671167+0000","last_undegraded":"2024-09-12T03:03:41.671167+0000","last_fullsized":"2024-09-12T03:03:41.671167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:19.053409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798384+0000","last_change":"2024-09-12T03:03:41.676526+0000","last_active":"2024-09-12T03:03:54.798384+0000","last_peered":"2024-09-12T03:03:54.798384+0000","last_clean":"2024-09-12T03:03:54.798384+0000","last_became_active":"2024-09-12T03:03:41.673301+0000","last_became_peered":"2024-09-12T03:03:41.673301+0000","last_unstale":"2024-09-12T03:03:54.798384+0000","last_undegraded":"2024-09-12T03:03:54.798384+0000","last_fullsized":"2024-09-12T03:03:54.798384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:04.648668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673644+0000","last_change":"2024-09-12T03:03:17.526846+0000","last_active":"2024-09-12T03:03:41.673644+0000","last_peered":"2024-09-12T03:03:41.673644+0000","last_clean":"2024-09-12T03:03:41.673644+0000","last_became_active":"2024-09-12T03:03:17.526412+0000","last_became_peered":"2024-09-12T03:03:17.526412+0000","last_unstale":"2024-09-12T03:03:41.673644+0000","last_undegraded":"2024-09-12T03:03:41.673644+0000","last_fullsized":"2024-09-12T03:03:41.673644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671567+0000","last_change":"2024-09-12T03:03:07.453989+0000","last_active":"2024-09-12T03:03:41.671567+0000","last_peered":"2024-09-12T03:03:41.671567+0000","last_clean":"2024-09-12T03:03:41.671567+0000","last_became_active":"2024-09-12T03:03:07.451796+0000","last_became_peered":"2024-09-12T03:03:07.451796+0000","last_unstale":"2024-09-12T03:03:41.671567+0000","last_undegraded":"2024-09-12T03:03:41.671567+0000","last_fullsized":"2024-09-12T03:03:41.671567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:27.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667160+0000","last_change":"2024-09-12T03:03:04.423756+0000","last_active":"2024-09-12T03:03:41.667160+0000","last_peered":"2024-09-12T03:03:41.667160+0000","last_clean":"2024-09-12T03:03:41.667160+0000","last_became_active":"2024-09-12T03:03:04.423075+0000","last_became_peered":"2024-09-12T03:03:04.423075+0000","last_unstale":"2024-09-12T03:03:41.667160+0000","last_undegraded":"2024-09-12T03:03:41.667160+0000","last_fullsized":"2024-09-12T03:03:41.667160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.201739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673954+0000","last_change":"2024-09-12T03:03:17.532906+0000","last_active":"2024-09-12T03:03:41.673954+0000","last_peered":"2024-09-12T03:03:41.673954+0000","last_clean":"2024-09-12T03:03:41.673954+0000","last_became_active":"2024-09-12T03:03:17.532343+0000","last_became_peered":"2024-09-12T03:03:17.532343+0000","last_unstale":"2024-09-12T03:03:41.673954+0000","last_undegraded":"2024-09-12T03:03:41.673954+0000","last_fullsized":"2024-09-12T03:03:41.673954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:05.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669111+0000","last_change":"2024-09-12T03:03:07.450375+0000","last_active":"2024-09-12T03:03:41.669111+0000","last_peered":"2024-09-12T03:03:41.669111+0000","last_clean":"2024-09-12T03:03:41.669111+0000","last_became_active":"2024-09-12T03:03:07.450061+0000","last_became_peered":"2024-09-12T03:03:07.450061+0000","last_unstale":"2024-09-12T03:03:41.669111+0000","last_undegraded":"2024-09-12T03:03:41.669111+0000","last_fullsized":"2024-09-12T03:03:41.669111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:05.922237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798743+0000","last_change":"2024-09-12T03:03:04.423664+0000","last_active":"2024-09-12T03:03:54.798743+0000","last_peered":"2024-09-12T03:03:54.798743+0000","last_clean":"2024-09-12T03:03:54.798743+0000","last_became_active":"2024-09-12T03:03:04.423285+0000","last_became_peered":"2024-09-12T03:03:04.423285+0000","last_unstale":"2024-09-12T03:03:54.798743+0000","last_undegraded":"2024-09-12T03:03:54.798743+0000","last_fullsized":"2024-09-12T03:03:54.798743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.782484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175832+0000","last_change":"2024-09-12T03:03:17.535385+0000","last_active":"2024-09-12T03:03:42.175832+0000","last_peered":"2024-09-12T03:03:42.175832+0000","last_clean":"2024-09-12T03:03:42.175832+0000","last_became_active":"2024-09-12T03:03:17.534775+0000","last_became_peered":"2024-09-12T03:03:17.534775+0000","last_unstale":"2024-09-12T03:03:42.175832+0000","last_undegraded":"2024-09-12T03:03:42.175832+0000","last_fullsized":"2024-09-12T03:03:42.175832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:02.549462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671807+0000","last_change":"2024-09-12T03:03:07.442786+0000","last_active":"2024-09-12T03:03:41.671807+0000","last_peered":"2024-09-12T03:03:41.671807+0000","last_clean":"2024-09-12T03:03:41.671807+0000","last_became_active":"2024-09-12T03:03:07.442456+0000","last_became_peered":"2024-09-12T03:03:07.442456+0000","last_unstale":"2024-09-12T03:03:41.671807+0000","last_undegraded":"2024-09-12T03:03:41.671807+0000","last_fullsized":"2024-09-12T03:03:41.671807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.882361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.799012+0000","last_change":"2024-09-12T03:03:41.675913+0000","last_active":"2024-09-12T03:03:54.799012+0000","last_peered":"2024-09-12T03:03:54.799012+0000","last_clean":"2024-09-12T03:03:54.799012+0000","last_became_active":"2024-09-12T03:03:41.675434+0000","last_became_peered":"2024-09-12T03:03:41.675434+0000","last_unstale":"2024-09-12T03:03:54.799012+0000","last_undegraded":"2024-09-12T03:03:54.799012+0000","last_fullsized":"2024-09-12T03:03:54.799012+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:36.484404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.177089+0000","last_change":"2024-09-12T03:03:42.177089+0000","last_active":"2024-09-12T03:03:42.177089+0000","last_peered":"2024-09-12T03:03:42.177089+0000","last_clean":"2024-09-12T03:03:42.177089+0000","last_became_active":"2024-09-12T03:03:42.176602+0000","last_became_peered":"2024-09-12T03:03:42.176602+0000","last_unstale":"2024-09-12T03:03:42.177089+0000","last_undegraded":"2024-09-12T03:03:42.177089+0000","last_fullsized":"2024-09-12T03:03:42.177089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:04.336826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670074+0000","last_change":"2024-09-12T03:03:07.457972+0000","last_active":"2024-09-12T03:03:41.670074+0000","last_peered":"2024-09-12T03:03:41.670074+0000","last_clean":"2024-09-12T03:03:41.670074+0000","last_became_active":"2024-09-12T03:03:07.457681+0000","last_became_peered":"2024-09-12T03:03:07.457681+0000","last_unstale":"2024-09-12T03:03:41.670074+0000","last_undegraded":"2024-09-12T03:03:41.670074+0000","last_fullsized":"2024-09-12T03:03:41.670074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:26.733432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.797099+0000","last_change":"2024-09-12T03:03:04.629721+0000","last_active":"2024-09-12T03:03:54.797099+0000","last_peered":"2024-09-12T03:03:54.797099+0000","last_clean":"2024-09-12T03:03:54.797099+0000","last_became_active":"2024-09-12T03:03:04.629362+0000","last_became_peered":"2024-09-12T03:03:04.629362+0000","last_unstale":"2024-09-12T03:03:54.797099+0000","last_undegraded":"2024-09-12T03:03:54.797099+0000","last_fullsized":"2024-09-12T03:03:54.797099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:09.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667128+0000","last_change":"2024-09-12T03:03:17.521048+0000","last_active":"2024-09-12T03:03:41.667128+0000","last_peered":"2024-09-12T03:03:41.667128+0000","last_clean":"2024-09-12T03:03:41.667128+0000","last_became_active":"2024-09-12T03:03:17.520640+0000","last_became_peered":"2024-09-12T03:03:17.520640+0000","last_unstale":"2024-09-12T03:03:41.667128+0000","last_undegraded":"2024-09-12T03:03:41.667128+0000","last_fullsized":"2024-09-12T03:03:41.667128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:06.243255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672329+0000","last_change":"2024-09-12T03:03:07.445497+0000","last_active":"2024-09-12T03:03:41.672329+0000","last_peered":"2024-09-12T03:03:41.672329+0000","last_clean":"2024-09-12T03:03:41.672329+0000","last_became_active":"2024-09-12T03:03:07.442245+0000","last_became_peered":"2024-09-12T03:03:07.442245+0000","last_unstale":"2024-09-12T03:03:41.672329+0000","last_undegraded":"2024-09-12T03:03:41.672329+0000","last_fullsized":"2024-09-12T03:03:41.672329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:13.147572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:10.464993+0000","last_change":"2024-09-12T03:03:04.424438+0000","last_active":"2024-09-12T03:04:10.464993+0000","last_peered":"2024-09-12T03:04:10.464993+0000","last_clean":"2024-09-12T03:04:10.464993+0000","last_became_active":"2024-09-12T03:03:04.424078+0000","last_became_peered":"2024-09-12T03:03:04.424078+0000","last_unstale":"2024-09-12T03:04:10.464993+0000","last_undegraded":"2024-09-12T03:04:10.464993+0000","last_fullsized":"2024-09-12T03:04:10.464993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:33.090082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671140+0000","last_change":"2024-09-12T03:03:17.532767+0000","last_active":"2024-09-12T03:03:41.671140+0000","last_peered":"2024-09-12T03:03:41.671140+0000","last_clean":"2024-09-12T03:03:41.671140+0000","last_became_active":"2024-09-12T03:03:17.532408+0000","last_became_peered":"2024-09-12T03:03:17.532408+0000","last_unstale":"2024-09-12T03:03:41.671140+0000","last_undegraded":"2024-09-12T03:03:41.671140+0000","last_fullsized":"2024-09-12T03:03:41.671140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:29.294605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953710+0000","last_change":"2024-09-12T03:03:07.448931+0000","last_active":"2024-09-12T03:03:41.953710+0000","last_peered":"2024-09-12T03:03:41.953710+0000","last_clean":"2024-09-12T03:03:41.953710+0000","last_became_active":"2024-09-12T03:03:07.447204+0000","last_became_peered":"2024-09-12T03:03:07.447204+0000","last_unstale":"2024-09-12T03:03:41.953710+0000","last_undegraded":"2024-09-12T03:03:41.953710+0000","last_fullsized":"2024-09-12T03:03:41.953710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:10.491591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.801007+0000","last_change":"2024-09-12T03:03:04.425805+0000","last_active":"2024-09-12T03:03:54.801007+0000","last_peered":"2024-09-12T03:03:54.801007+0000","last_clean":"2024-09-12T03:03:54.801007+0000","last_became_active":"2024-09-12T03:03:04.425469+0000","last_became_peered":"2024-09-12T03:03:04.425469+0000","last_unstale":"2024-09-12T03:03:54.801007+0000","last_undegraded":"2024-09-12T03:03:54.801007+0000","last_fullsized":"2024-09-12T03:03:54.801007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:27.976934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669627+0000","last_change":"2024-09-12T03:03:17.525537+0000","last_active":"2024-09-12T03:03:41.669627+0000","last_peered":"2024-09-12T03:03:41.669627+0000","last_clean":"2024-09-12T03:03:41.669627+0000","last_became_active":"2024-09-12T03:03:17.525030+0000","last_became_peered":"2024-09-12T03:03:17.525030+0000","last_unstale":"2024-09-12T03:03:41.669627+0000","last_undegraded":"2024-09-12T03:03:41.669627+0000","last_fullsized":"2024-09-12T03:03:41.669627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:53.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.876773+0000","last_change":"2024-09-12T03:03:07.443712+0000","last_active":"2024-09-12T03:03:41.876773+0000","last_peered":"2024-09-12T03:03:41.876773+0000","last_clean":"2024-09-12T03:03:41.876773+0000","last_became_active":"2024-09-12T03:03:07.443374+0000","last_became_peered":"2024-09-12T03:03:07.443374+0000","last_unstale":"2024-09-12T03:03:41.876773+0000","last_undegraded":"2024-09-12T03:03:41.876773+0000","last_fullsized":"2024-09-12T03:03:41.876773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:09.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175945+0000","last_change":"2024-09-12T03:03:04.427683+0000","last_active":"2024-09-12T03:03:42.175945+0000","last_peered":"2024-09-12T03:03:42.175945+0000","last_clean":"2024-09-12T03:03:42.175945+0000","last_became_active":"2024-09-12T03:03:04.427276+0000","last_became_peered":"2024-09-12T03:03:04.427276+0000","last_unstale":"2024-09-12T03:03:42.175945+0000","last_undegraded":"2024-09-12T03:03:42.175945+0000","last_fullsized":"2024-09-12T03:03:42.175945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:44.540369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.878110+0000","last_change":"2024-09-12T03:03:41.878110+0000","last_active":"2024-09-12T03:03:41.878110+0000","last_peered":"2024-09-12T03:03:41.878110+0000","last_clean":"2024-09-12T03:03:41.878110+0000","last_became_active":"2024-09-12T03:03:41.877719+0000","last_became_peered":"2024-09-12T03:03:41.877719+0000","last_unstale":"2024-09-12T03:03:41.878110+0000","last_undegraded":"2024-09-12T03:03:41.878110+0000","last_fullsized":"2024-09-12T03:03:41.878110+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:08.312660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.876744+0000","last_change":"2024-09-12T03:03:07.454093+0000","last_active":"2024-09-12T03:03:41.876744+0000","last_peered":"2024-09-12T03:03:41.876744+0000","last_clean":"2024-09-12T03:03:41.876744+0000","last_became_active":"2024-09-12T03:03:07.453676+0000","last_became_peered":"2024-09-12T03:03:07.453676+0000","last_unstale":"2024-09-12T03:03:41.876744+0000","last_undegraded":"2024-09-12T03:03:41.876744+0000","last_fullsized":"2024-09-12T03:03:41.876744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:18.045572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671400+0000","last_change":"2024-09-12T03:03:04.425336+0000","last_active":"2024-09-12T03:03:41.671400+0000","last_peered":"2024-09-12T03:03:41.671400+0000","last_clean":"2024-09-12T03:03:41.671400+0000","last_became_active":"2024-09-12T03:03:04.424890+0000","last_became_peered":"2024-09-12T03:03:04.424890+0000","last_unstale":"2024-09-12T03:03:41.671400+0000","last_undegraded":"2024-09-12T03:03:41.671400+0000","last_fullsized":"2024-09-12T03:03:41.671400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.443627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953494+0000","last_change":"2024-09-12T03:03:17.535300+0000","last_active":"2024-09-12T03:03:41.953494+0000","last_peered":"2024-09-12T03:03:41.953494+0000","last_clean":"2024-09-12T03:03:41.953494+0000","last_became_active":"2024-09-12T03:03:17.534937+0000","last_became_peered":"2024-09-12T03:03:17.534937+0000","last_unstale":"2024-09-12T03:03:41.953494+0000","last_undegraded":"2024-09-12T03:03:41.953494+0000","last_fullsized":"2024-09-12T03:03:41.953494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:54.874973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454670+0000","last_change":"2024-09-12T03:03:07.453964+0000","last_active":"2024-09-12T03:03:42.454670+0000","last_peered":"2024-09-12T03:03:42.454670+0000","last_clean":"2024-09-12T03:03:42.454670+0000","last_became_active":"2024-09-12T03:03:07.453622+0000","last_became_peered":"2024-09-12T03:03:07.453622+0000","last_unstale":"2024-09-12T03:03:42.454670+0000","last_undegraded":"2024-09-12T03:03:42.454670+0000","last_fullsized":"2024-09-12T03:03:42.454670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:17.452519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:09.832901+0000","last_change":"2024-09-12T03:03:04.428526+0000","last_active":"2024-09-12T03:04:09.832901+0000","last_peered":"2024-09-12T03:04:09.832901+0000","last_clean":"2024-09-12T03:04:09.832901+0000","last_became_active":"2024-09-12T03:03:04.428221+0000","last_became_peered":"2024-09-12T03:03:04.428221+0000","last_unstale":"2024-09-12T03:04:09.832901+0000","last_undegraded":"2024-09-12T03:04:09.832901+0000","last_fullsized":"2024-09-12T03:04:09.832901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:50.999346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953812+0000","last_change":"2024-09-12T03:03:17.535146+0000","last_active":"2024-09-12T03:03:41.953812+0000","last_peered":"2024-09-12T03:03:41.953812+0000","last_clean":"2024-09-12T03:03:41.953812+0000","last_became_active":"2024-09-12T03:03:17.534677+0000","last_became_peered":"2024-09-12T03:03:17.534677+0000","last_unstale":"2024-09-12T03:03:41.953812+0000","last_undegraded":"2024-09-12T03:03:41.953812+0000","last_fullsized":"2024-09-12T03:03:41.953812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:09.874993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953220+0000","last_change":"2024-09-12T03:03:07.431348+0000","last_active":"2024-09-12T03:03:41.953220+0000","last_peered":"2024-09-12T03:03:41.953220+0000","last_clean":"2024-09-12T03:03:41.953220+0000","last_became_active":"2024-09-12T03:03:07.430912+0000","last_became_peered":"2024-09-12T03:03:07.430912+0000","last_unstale":"2024-09-12T03:03:41.953220+0000","last_undegraded":"2024-09-12T03:03:41.953220+0000","last_fullsized":"2024-09-12T03:03:41.953220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:27.644324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807071+0000","last_change":"2024-09-12T03:03:42.456245+0000","last_active":"2024-09-12T03:03:55.807071+0000","last_peered":"2024-09-12T03:03:55.807071+0000","last_clean":"2024-09-12T03:03:55.807071+0000","last_became_active":"2024-09-12T03:03:42.455812+0000","last_became_peered":"2024-09-12T03:03:42.455812+0000","last_unstale":"2024-09-12T03:03:55.807071+0000","last_undegraded":"2024-09-12T03:03:55.807071+0000","last_fullsized":"2024-09-12T03:03:55.807071+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454997+0000","last_change":"2024-09-12T03:03:17.527371+0000","last_active":"2024-09-12T03:03:42.454997+0000","last_peered":"2024-09-12T03:03:42.454997+0000","last_clean":"2024-09-12T03:03:42.454997+0000","last_became_active":"2024-09-12T03:03:17.526818+0000","last_became_peered":"2024-09-12T03:03:17.526818+0000","last_unstale":"2024-09-12T03:03:42.454997+0000","last_undegraded":"2024-09-12T03:03:42.454997+0000","last_fullsized":"2024-09-12T03:03:42.454997+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:49.519942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671585+0000","last_change":"2024-09-12T03:03:07.451364+0000","last_active":"2024-09-12T03:03:41.671585+0000","last_peered":"2024-09-12T03:03:41.671585+0000","last_clean":"2024-09-12T03:03:41.671585+0000","last_became_active":"2024-09-12T03:03:07.450964+0000","last_became_peered":"2024-09-12T03:03:07.450964+0000","last_unstale":"2024-09-12T03:03:41.671585+0000","last_undegraded":"2024-09-12T03:03:41.671585+0000","last_fullsized":"2024-09-12T03:03:41.671585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:56.811698+0000","last_change":"2024-09-12T03:03:04.430485+0000","last_active":"2024-09-12T03:03:56.811698+0000","last_peered":"2024-09-12T03:03:56.811698+0000","last_clean":"2024-09-12T03:03:56.811698+0000","last_became_active":"2024-09-12T03:03:04.430096+0000","last_became_peered":"2024-09-12T03:03:04.430096+0000","last_unstale":"2024-09-12T03:03:56.811698+0000","last_undegraded":"2024-09-12T03:03:56.811698+0000","last_fullsized":"2024-09-12T03:03:56.811698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:26.486223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670839+0000","last_change":"2024-09-12T03:01:48.369350+0000","last_active":"2024-09-12T03:03:41.670839+0000","last_peered":"2024-09-12T03:03:41.670839+0000","last_clean":"2024-09-12T03:03:41.670839+0000","last_became_active":"2024-09-12T03:01:48.369011+0000","last_became_peered":"2024-09-12T03:01:48.369011+0000","last_unstale":"2024-09-12T03:03:41.670839+0000","last_undegraded":"2024-09-12T03:03:41.670839+0000","last_fullsized":"2024-09-12T03:03:41.670839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_clean_scrub_stamp":"2024-09-12T02:58:06.597228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:57.348343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672013+0000","last_change":"2024-09-12T03:03:17.541157+0000","last_active":"2024-09-12T03:03:41.672013+0000","last_peered":"2024-09-12T03:03:41.672013+0000","last_clean":"2024-09-12T03:03:41.672013+0000","last_became_active":"2024-09-12T03:03:17.540687+0000","last_became_peered":"2024-09-12T03:03:17.540687+0000","last_unstale":"2024-09-12T03:03:41.672013+0000","last_undegraded":"2024-09-12T03:03:41.672013+0000","last_fullsized":"2024-09-12T03:03:41.672013+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:35.072323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672214+0000","last_change":"2024-09-12T03:03:07.445674+0000","last_active":"2024-09-12T03:03:41.672214+0000","last_peered":"2024-09-12T03:03:41.672214+0000","last_clean":"2024-09-12T03:03:41.672214+0000","last_became_active":"2024-09-12T03:03:07.445237+0000","last_became_peered":"2024-09-12T03:03:07.445237+0000","last_unstale":"2024-09-12T03:03:41.672214+0000","last_undegraded":"2024-09-12T03:03:41.672214+0000","last_fullsized":"2024-09-12T03:03:41.672214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:56.876238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.810512+0000","last_change":"2024-09-12T03:03:04.629794+0000","last_active":"2024-09-12T03:03:55.810512+0000","last_peered":"2024-09-12T03:03:55.810512+0000","last_clean":"2024-09-12T03:03:55.810512+0000","last_became_active":"2024-09-12T03:03:04.629352+0000","last_became_peered":"2024-09-12T03:03:04.629352+0000","last_unstale":"2024-09-12T03:03:55.810512+0000","last_undegraded":"2024-09-12T03:03:55.810512+0000","last_fullsized":"2024-09-12T03:03:55.810512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:32.168567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672261+0000","last_change":"2024-09-12T03:03:17.525419+0000","last_active":"2024-09-12T03:03:41.672261+0000","last_peered":"2024-09-12T03:03:41.672261+0000","last_clean":"2024-09-12T03:03:41.672261+0000","last_became_active":"2024-09-12T03:03:17.524894+0000","last_became_peered":"2024-09-12T03:03:17.524894+0000","last_unstale":"2024-09-12T03:03:41.672261+0000","last_undegraded":"2024-09-12T03:03:41.672261+0000","last_fullsized":"2024-09-12T03:03:41.672261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:57.441301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671959+0000","last_change":"2024-09-12T03:03:07.447990+0000","last_active":"2024-09-12T03:03:41.671959+0000","last_peered":"2024-09-12T03:03:41.671959+0000","last_clean":"2024-09-12T03:03:41.671959+0000","last_became_active":"2024-09-12T03:03:07.447648+0000","last_became_peered":"2024-09-12T03:03:07.447648+0000","last_unstale":"2024-09-12T03:03:41.671959+0000","last_undegraded":"2024-09-12T03:03:41.671959+0000","last_fullsized":"2024-09-12T03:03:41.671959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:48.639273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455107+0000","last_change":"2024-09-12T03:03:04.421306+0000","last_active":"2024-09-12T03:03:42.455107+0000","last_peered":"2024-09-12T03:03:42.455107+0000","last_clean":"2024-09-12T03:03:42.455107+0000","last_became_active":"2024-09-12T03:03:04.420688+0000","last_became_peered":"2024-09-12T03:03:04.420688+0000","last_unstale":"2024-09-12T03:03:42.455107+0000","last_undegraded":"2024-09-12T03:03:42.455107+0000","last_fullsized":"2024-09-12T03:03:42.455107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:15.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670487+0000","last_change":"2024-09-12T03:03:17.530457+0000","last_active":"2024-09-12T03:03:41.670487+0000","last_peered":"2024-09-12T03:03:41.670487+0000","last_clean":"2024-09-12T03:03:41.670487+0000","last_became_active":"2024-09-12T03:03:17.530044+0000","last_became_peered":"2024-09-12T03:03:17.530044+0000","last_unstale":"2024-09-12T03:03:41.670487+0000","last_undegraded":"2024-09-12T03:03:41.670487+0000","last_fullsized":"2024-09-12T03:03:41.670487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:01.304089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672473+0000","last_change":"2024-09-12T03:03:07.445935+0000","last_active":"2024-09-12T03:03:41.672473+0000","last_peered":"2024-09-12T03:03:41.672473+0000","last_clean":"2024-09-12T03:03:41.672473+0000","last_became_active":"2024-09-12T03:03:07.444960+0000","last_became_peered":"2024-09-12T03:03:07.444960+0000","last_unstale":"2024-09-12T03:03:41.672473+0000","last_undegraded":"2024-09-12T03:03:41.672473+0000","last_fullsized":"2024-09-12T03:03:41.672473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:53.246923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953867+0000","last_change":"2024-09-12T03:03:04.424147+0000","last_active":"2024-09-12T03:03:41.953867+0000","last_peered":"2024-09-12T03:03:41.953867+0000","last_clean":"2024-09-12T03:03:41.953867+0000","last_became_active":"2024-09-12T03:03:04.423787+0000","last_became_peered":"2024-09-12T03:03:04.423787+0000","last_unstale":"2024-09-12T03:03:41.953867+0000","last_undegraded":"2024-09-12T03:03:41.953867+0000","last_fullsized":"2024-09-12T03:03:41.953867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:48.155556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454133+0000","last_change":"2024-09-12T03:03:17.520974+0000","last_active":"2024-09-12T03:03:42.454133+0000","last_peered":"2024-09-12T03:03:42.454133+0000","last_clean":"2024-09-12T03:03:42.454133+0000","last_became_active":"2024-09-12T03:03:17.520332+0000","last_became_peered":"2024-09-12T03:03:17.520332+0000","last_unstale":"2024-09-12T03:03:42.454133+0000","last_undegraded":"2024-09-12T03:03:42.454133+0000","last_fullsized":"2024-09-12T03:03:42.454133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:45.061197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456589+0000","last_change":"2024-09-12T03:03:42.456589+0000","last_active":"2024-09-12T03:03:42.456589+0000","last_peered":"2024-09-12T03:03:42.456589+0000","last_clean":"2024-09-12T03:03:42.456589+0000","last_became_active":"2024-09-12T03:03:42.456147+0000","last_became_peered":"2024-09-12T03:03:42.456147+0000","last_unstale":"2024-09-12T03:03:42.456589+0000","last_undegraded":"2024-09-12T03:03:42.456589+0000","last_fullsized":"2024-09-12T03:03:42.456589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:58.009928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:56.804513+0000","last_change":"2024-09-12T03:03:04.420471+0000","last_active":"2024-09-12T03:03:56.804513+0000","last_peered":"2024-09-12T03:03:56.804513+0000","last_clean":"2024-09-12T03:03:56.804513+0000","last_became_active":"2024-09-12T03:03:04.420093+0000","last_became_peered":"2024-09-12T03:03:04.420093+0000","last_unstale":"2024-09-12T03:03:56.804513+0000","last_undegraded":"2024-09-12T03:03:56.804513+0000","last_fullsized":"2024-09-12T03:03:56.804513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:23.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454371+0000","last_change":"2024-09-12T03:03:17.528073+0000","last_active":"2024-09-12T03:03:42.454371+0000","last_peered":"2024-09-12T03:03:42.454371+0000","last_clean":"2024-09-12T03:03:42.454371+0000","last_became_active":"2024-09-12T03:03:17.527654+0000","last_became_peered":"2024-09-12T03:03:17.527654+0000","last_unstale":"2024-09-12T03:03:42.454371+0000","last_undegraded":"2024-09-12T03:03:42.454371+0000","last_fullsized":"2024-09-12T03:03:42.454371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:45.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454660+0000","last_change":"2024-09-12T03:03:07.455305+0000","last_active":"2024-09-12T03:03:42.454660+0000","last_peered":"2024-09-12T03:03:42.454660+0000","last_clean":"2024-09-12T03:03:42.454660+0000","last_became_active":"2024-09-12T03:03:07.454957+0000","last_became_peered":"2024-09-12T03:03:07.454957+0000","last_unstale":"2024-09-12T03:03:42.454660+0000","last_undegraded":"2024-09-12T03:03:42.454660+0000","last_fullsized":"2024-09-12T03:03:42.454660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:07.152100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809053+0000","last_change":"2024-09-12T03:03:04.424531+0000","last_active":"2024-09-12T03:03:55.809053+0000","last_peered":"2024-09-12T03:03:55.809053+0000","last_clean":"2024-09-12T03:03:55.809053+0000","last_became_active":"2024-09-12T03:03:04.424230+0000","last_became_peered":"2024-09-12T03:03:04.424230+0000","last_unstale":"2024-09-12T03:03:55.809053+0000","last_undegraded":"2024-09-12T03:03:55.809053+0000","last_fullsized":"2024-09-12T03:03:55.809053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:01.392262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953859+0000","last_change":"2024-09-12T03:03:17.532131+0000","last_active":"2024-09-12T03:03:41.953859+0000","last_peered":"2024-09-12T03:03:41.953859+0000","last_clean":"2024-09-12T03:03:41.953859+0000","last_became_active":"2024-09-12T03:03:17.531711+0000","last_became_peered":"2024-09-12T03:03:17.531711+0000","last_unstale":"2024-09-12T03:03:41.953859+0000","last_undegraded":"2024-09-12T03:03:41.953859+0000","last_fullsized":"2024-09-12T03:03:41.953859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:15.880954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670194+0000","last_change":"2024-09-12T03:03:07.449101+0000","last_active":"2024-09-12T03:03:41.670194+0000","last_peered":"2024-09-12T03:03:41.670194+0000","last_clean":"2024-09-12T03:03:41.670194+0000","last_became_active":"2024-09-12T03:03:07.448719+0000","last_became_peered":"2024-09-12T03:03:07.448719+0000","last_unstale":"2024-09-12T03:03:41.670194+0000","last_undegraded":"2024-09-12T03:03:41.670194+0000","last_fullsized":"2024-09-12T03:03:41.670194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:45.400939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:10.835515+0000","last_change":"2024-09-12T03:03:04.422174+0000","last_active":"2024-09-12T03:04:10.835515+0000","last_peered":"2024-09-12T03:04:10.835515+0000","last_clean":"2024-09-12T03:04:10.835515+0000","last_became_active":"2024-09-12T03:03:04.421771+0000","last_became_peered":"2024-09-12T03:03:04.421771+0000","last_unstale":"2024-09-12T03:04:10.835515+0000","last_undegraded":"2024-09-12T03:04:10.835515+0000","last_fullsized":"2024-09-12T03:04:10.835515+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:38.149590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175926+0000","last_change":"2024-09-12T03:03:17.535123+0000","last_active":"2024-09-12T03:03:42.175926+0000","last_peered":"2024-09-12T03:03:42.175926+0000","last_clean":"2024-09-12T03:03:42.175926+0000","last_became_active":"2024-09-12T03:03:17.534218+0000","last_became_peered":"2024-09-12T03:03:17.534218+0000","last_unstale":"2024-09-12T03:03:42.175926+0000","last_undegraded":"2024-09-12T03:03:42.175926+0000","last_fullsized":"2024-09-12T03:03:42.175926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:55.870690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671534+0000","last_change":"2024-09-12T03:03:07.453736+0000","last_active":"2024-09-12T03:03:41.671534+0000","last_peered":"2024-09-12T03:03:41.671534+0000","last_clean":"2024-09-12T03:03:41.671534+0000","last_became_active":"2024-09-12T03:03:07.453404+0000","last_became_peered":"2024-09-12T03:03:07.453404+0000","last_unstale":"2024-09-12T03:03:41.671534+0000","last_undegraded":"2024-09-12T03:03:41.671534+0000","last_fullsized":"2024-09-12T03:03:41.671534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:57.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798006+0000","last_change":"2024-09-12T03:03:04.419663+0000","last_active":"2024-09-12T03:03:54.798006+0000","last_peered":"2024-09-12T03:03:54.798006+0000","last_clean":"2024-09-12T03:03:54.798006+0000","last_became_active":"2024-09-12T03:03:04.419119+0000","last_became_peered":"2024-09-12T03:03:04.419119+0000","last_unstale":"2024-09-12T03:03:54.798006+0000","last_undegraded":"2024-09-12T03:03:54.798006+0000","last_fullsized":"2024-09-12T03:03:54.798006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:11.809683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669223+0000","last_change":"2024-09-12T03:03:17.532686+0000","last_active":"2024-09-12T03:03:41.669223+0000","last_peered":"2024-09-12T03:03:41.669223+0000","last_clean":"2024-09-12T03:03:41.669223+0000","last_became_active":"2024-09-12T03:03:17.532234+0000","last_became_peered":"2024-09-12T03:03:17.532234+0000","last_unstale":"2024-09-12T03:03:41.669223+0000","last_undegraded":"2024-09-12T03:03:41.669223+0000","last_fullsized":"2024-09-12T03:03:41.669223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:49.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953501+0000","last_change":"2024-09-12T03:03:07.441911+0000","last_active":"2024-09-12T03:03:41.953501+0000","last_peered":"2024-09-12T03:03:41.953501+0000","last_clean":"2024-09-12T03:03:41.953501+0000","last_became_active":"2024-09-12T03:03:07.441323+0000","last_became_peered":"2024-09-12T03:03:07.441323+0000","last_unstale":"2024-09-12T03:03:41.953501+0000","last_undegraded":"2024-09-12T03:03:41.953501+0000","last_fullsized":"2024-09-12T03:03:41.953501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:03.830843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.810725+0000","last_change":"2024-09-12T03:03:04.423648+0000","last_active":"2024-09-12T03:03:55.810725+0000","last_peered":"2024-09-12T03:03:55.810725+0000","last_clean":"2024-09-12T03:03:55.810725+0000","last_became_active":"2024-09-12T03:03:04.423274+0000","last_became_peered":"2024-09-12T03:03:04.423274+0000","last_unstale":"2024-09-12T03:03:55.810725+0000","last_undegraded":"2024-09-12T03:03:55.810725+0000","last_fullsized":"2024-09-12T03:03:55.810725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:50.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454771+0000","last_change":"2024-09-12T03:03:17.522286+0000","last_active":"2024-09-12T03:03:42.454771+0000","last_peered":"2024-09-12T03:03:42.454771+0000","last_clean":"2024-09-12T03:03:42.454771+0000","last_became_active":"2024-09-12T03:03:17.521909+0000","last_became_peered":"2024-09-12T03:03:17.521909+0000","last_unstale":"2024-09-12T03:03:42.454771+0000","last_undegraded":"2024-09-12T03:03:42.454771+0000","last_fullsized":"2024-09-12T03:03:42.454771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:13.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672284+0000","last_change":"2024-09-12T03:03:07.445445+0000","last_active":"2024-09-12T03:03:41.672284+0000","last_peered":"2024-09-12T03:03:41.672284+0000","last_clean":"2024-09-12T03:03:41.672284+0000","last_became_active":"2024-09-12T03:03:07.445086+0000","last_became_peered":"2024-09-12T03:03:07.445086+0000","last_unstale":"2024-09-12T03:03:41.672284+0000","last_undegraded":"2024-09-12T03:03:41.672284+0000","last_fullsized":"2024-09-12T03:03:41.672284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:01.046168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'29","reported_seq":114,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:14.840850+0000","last_change":"2024-09-12T03:03:04.425545+0000","last_active":"2024-09-12T03:04:14.840850+0000","last_peered":"2024-09-12T03:04:14.840850+0000","last_clean":"2024-09-12T03:04:14.840850+0000","last_became_active":"2024-09-12T03:03:04.425141+0000","last_became_peered":"2024-09-12T03:03:04.425141+0000","last_unstale":"2024-09-12T03:04:14.840850+0000","last_undegraded":"2024-09-12T03:04:14.840850+0000","last_fullsized":"2024-09-12T03:04:14.840850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:54.548787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57258,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":78,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175605+0000","last_change":"2024-09-12T03:03:17.538157+0000","last_active":"2024-09-12T03:03:42.175605+0000","last_peered":"2024-09-12T03:03:42.175605+0000","last_clean":"2024-09-12T03:03:42.175605+0000","last_became_active":"2024-09-12T03:03:17.537194+0000","last_became_peered":"2024-09-12T03:03:17.537194+0000","last_unstale":"2024-09-12T03:03:42.175605+0000","last_undegraded":"2024-09-12T03:03:42.175605+0000","last_fullsized":"2024-09-12T03:03:42.175605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:00.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672227+0000","last_change":"2024-09-12T03:03:07.446470+0000","last_active":"2024-09-12T03:03:41.672227+0000","last_peered":"2024-09-12T03:03:41.672227+0000","last_clean":"2024-09-12T03:03:41.672227+0000","last_became_active":"2024-09-12T03:03:07.445975+0000","last_became_peered":"2024-09-12T03:03:07.445975+0000","last_unstale":"2024-09-12T03:03:41.672227+0000","last_undegraded":"2024-09-12T03:03:41.672227+0000","last_fullsized":"2024-09-12T03:03:41.672227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:47.795117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.799739+0000","last_change":"2024-09-12T03:03:04.421833+0000","last_active":"2024-09-12T03:03:54.799739+0000","last_peered":"2024-09-12T03:03:54.799739+0000","last_clean":"2024-09-12T03:03:54.799739+0000","last_became_active":"2024-09-12T03:03:04.421439+0000","last_became_peered":"2024-09-12T03:03:04.421439+0000","last_unstale":"2024-09-12T03:03:54.799739+0000","last_undegraded":"2024-09-12T03:03:54.799739+0000","last_fullsized":"2024-09-12T03:03:54.799739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673561+0000","last_change":"2024-09-12T03:03:17.541327+0000","last_active":"2024-09-12T03:03:41.673561+0000","last_peered":"2024-09-12T03:03:41.673561+0000","last_clean":"2024-09-12T03:03:41.673561+0000","last_became_active":"2024-09-12T03:03:17.540959+0000","last_became_peered":"2024-09-12T03:03:17.540959+0000","last_unstale":"2024-09-12T03:03:41.673561+0000","last_undegraded":"2024-09-12T03:03:41.673561+0000","last_fullsized":"2024-09-12T03:03:41.673561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:16.833511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670609+0000","last_change":"2024-09-12T03:03:07.458806+0000","last_active":"2024-09-12T03:03:41.670609+0000","last_peered":"2024-09-12T03:03:41.670609+0000","last_clean":"2024-09-12T03:03:41.670609+0000","last_became_active":"2024-09-12T03:03:07.458501+0000","last_became_peered":"2024-09-12T03:03:07.458501+0000","last_unstale":"2024-09-12T03:03:41.670609+0000","last_undegraded":"2024-09-12T03:03:41.670609+0000","last_fullsized":"2024-09-12T03:03:41.670609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:54.608552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.797761+0000","last_change":"2024-09-12T03:03:41.676350+0000","last_active":"2024-09-12T03:03:54.797761+0000","last_peered":"2024-09-12T03:03:54.797761+0000","last_clean":"2024-09-12T03:03:54.797761+0000","last_became_active":"2024-09-12T03:03:41.675935+0000","last_became_peered":"2024-09-12T03:03:41.675935+0000","last_unstale":"2024-09-12T03:03:54.797761+0000","last_undegraded":"2024-09-12T03:03:54.797761+0000","last_fullsized":"2024-09-12T03:03:54.797761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:11.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175316+0000","last_change":"2024-09-12T03:03:17.533670+0000","last_active":"2024-09-12T03:03:42.175316+0000","last_peered":"2024-09-12T03:03:42.175316+0000","last_clean":"2024-09-12T03:03:42.175316+0000","last_became_active":"2024-09-12T03:03:17.530516+0000","last_became_peered":"2024-09-12T03:03:17.530516+0000","last_unstale":"2024-09-12T03:03:42.175316+0000","last_undegraded":"2024-09-12T03:03:42.175316+0000","last_fullsized":"2024-09-12T03:03:42.175316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:02.228503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672465+0000","last_change":"2024-09-12T03:03:07.448760+0000","last_active":"2024-09-12T03:03:41.672465+0000","last_peered":"2024-09-12T03:03:41.672465+0000","last_clean":"2024-09-12T03:03:41.672465+0000","last_became_active":"2024-09-12T03:03:07.447978+0000","last_became_peered":"2024-09-12T03:03:07.447978+0000","last_unstale":"2024-09-12T03:03:41.672465+0000","last_undegraded":"2024-09-12T03:03:41.672465+0000","last_fullsized":"2024-09-12T03:03:41.672465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.319881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":62,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:13.839379+0000","last_change":"2024-09-12T03:03:04.427752+0000","last_active":"2024-09-12T03:04:13.839379+0000","last_peered":"2024-09-12T03:04:13.839379+0000","last_clean":"2024-09-12T03:04:13.839379+0000","last_became_active":"2024-09-12T03:03:04.427327+0000","last_became_peered":"2024-09-12T03:03:04.427327+0000","last_unstale":"2024-09-12T03:04:13.839379+0000","last_undegraded":"2024-09-12T03:04:13.839379+0000","last_fullsized":"2024-09-12T03:04:13.839379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:51.271870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671398+0000","last_change":"2024-09-12T03:03:17.520463+0000","last_active":"2024-09-12T03:03:41.671398+0000","last_peered":"2024-09-12T03:03:41.671398+0000","last_clean":"2024-09-12T03:03:41.671398+0000","last_became_active":"2024-09-12T03:03:17.520081+0000","last_became_peered":"2024-09-12T03:03:17.520081+0000","last_unstale":"2024-09-12T03:03:41.671398+0000","last_undegraded":"2024-09-12T03:03:41.671398+0000","last_fullsized":"2024-09-12T03:03:41.671398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:56.707115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669877+0000","last_change":"2024-09-12T03:03:17.532373+0000","last_active":"2024-09-12T03:03:41.669877+0000","last_peered":"2024-09-12T03:03:41.669877+0000","last_clean":"2024-09-12T03:03:41.669877+0000","last_became_active":"2024-09-12T03:03:17.525425+0000","last_became_peered":"2024-09-12T03:03:17.525425+0000","last_unstale":"2024-09-12T03:03:41.669877+0000","last_undegraded":"2024-09-12T03:03:41.669877+0000","last_fullsized":"2024-09-12T03:03:41.669877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:29.811678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175798+0000","last_change":"2024-09-12T03:03:07.445936+0000","last_active":"2024-09-12T03:03:42.175798+0000","last_peered":"2024-09-12T03:03:42.175798+0000","last_clean":"2024-09-12T03:03:42.175798+0000","last_became_active":"2024-09-12T03:03:07.445643+0000","last_became_peered":"2024-09-12T03:03:07.445643+0000","last_unstale":"2024-09-12T03:03:42.175798+0000","last_undegraded":"2024-09-12T03:03:42.175798+0000","last_fullsized":"2024-09-12T03:03:42.175798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:07.477841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455384+0000","last_change":"2024-09-12T03:03:04.424947+0000","last_active":"2024-09-12T03:03:42.455384+0000","last_peered":"2024-09-12T03:03:42.455384+0000","last_clean":"2024-09-12T03:03:42.455384+0000","last_became_active":"2024-09-12T03:03:04.424297+0000","last_became_peered":"2024-09-12T03:03:04.424297+0000","last_unstale":"2024-09-12T03:03:42.455384+0000","last_undegraded":"2024-09-12T03:03:42.455384+0000","last_fullsized":"2024-09-12T03:03:42.455384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:11.187945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669389+0000","last_change":"2024-09-12T03:03:17.530616+0000","last_active":"2024-09-12T03:03:41.669389+0000","last_peered":"2024-09-12T03:03:41.669389+0000","last_clean":"2024-09-12T03:03:41.669389+0000","last_became_active":"2024-09-12T03:03:17.530146+0000","last_became_peered":"2024-09-12T03:03:17.530146+0000","last_unstale":"2024-09-12T03:03:41.669389+0000","last_undegraded":"2024-09-12T03:03:41.669389+0000","last_fullsized":"2024-09-12T03:03:41.669389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:26.892311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671245+0000","last_change":"2024-09-12T03:03:07.448084+0000","last_active":"2024-09-12T03:03:41.671245+0000","last_peered":"2024-09-12T03:03:41.671245+0000","last_clean":"2024-09-12T03:03:41.671245+0000","last_became_active":"2024-09-12T03:03:07.447363+0000","last_became_peered":"2024-09-12T03:03:07.447363+0000","last_unstale":"2024-09-12T03:03:41.671245+0000","last_undegraded":"2024-09-12T03:03:41.671245+0000","last_fullsized":"2024-09-12T03:03:41.671245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:07.451892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809610+0000","last_change":"2024-09-12T03:03:04.430371+0000","last_active":"2024-09-12T03:03:55.809610+0000","last_peered":"2024-09-12T03:03:55.809610+0000","last_clean":"2024-09-12T03:03:55.809610+0000","last_became_active":"2024-09-12T03:03:04.429811+0000","last_became_peered":"2024-09-12T03:03:04.429811+0000","last_unstale":"2024-09-12T03:03:55.809610+0000","last_undegraded":"2024-09-12T03:03:55.809610+0000","last_fullsized":"2024-09-12T03:03:55.809610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:07.443242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807883+0000","last_change":"2024-09-12T03:03:04.422909+0000","last_active":"2024-09-12T03:03:55.807883+0000","last_peered":"2024-09-12T03:03:55.807883+0000","last_clean":"2024-09-12T03:03:55.807883+0000","last_became_active":"2024-09-12T03:03:04.422476+0000","last_became_peered":"2024-09-12T03:03:04.422476+0000","last_unstale":"2024-09-12T03:03:55.807883+0000","last_undegraded":"2024-09-12T03:03:55.807883+0000","last_fullsized":"2024-09-12T03:03:55.807883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455001+0000","last_change":"2024-09-12T03:03:07.454873+0000","last_active":"2024-09-12T03:03:42.455001+0000","last_peered":"2024-09-12T03:03:42.455001+0000","last_clean":"2024-09-12T03:03:42.455001+0000","last_became_active":"2024-09-12T03:03:07.454521+0000","last_became_peered":"2024-09-12T03:03:07.454521+0000","last_unstale":"2024-09-12T03:03:42.455001+0000","last_undegraded":"2024-09-12T03:03:42.455001+0000","last_fullsized":"2024-09-12T03:03:42.455001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:01.710362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456398+0000","last_change":"2024-09-12T03:03:42.456398+0000","last_active":"2024-09-12T03:03:42.456398+0000","last_peered":"2024-09-12T03:03:42.456398+0000","last_clean":"2024-09-12T03:03:42.456398+0000","last_became_active":"2024-09-12T03:03:42.455878+0000","last_became_peered":"2024-09-12T03:03:42.455878+0000","last_unstale":"2024-09-12T03:03:42.456398+0000","last_undegraded":"2024-09-12T03:03:42.456398+0000","last_fullsized":"2024-09-12T03:03:42.456398+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:31.760018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:09.833566+0000","last_change":"2024-09-12T03:03:04.424039+0000","last_active":"2024-09-12T03:04:09.833566+0000","last_peered":"2024-09-12T03:04:09.833566+0000","last_clean":"2024-09-12T03:04:09.833566+0000","last_became_active":"2024-09-12T03:03:04.422184+0000","last_became_peered":"2024-09-12T03:03:04.422184+0000","last_unstale":"2024-09-12T03:04:09.833566+0000","last_undegraded":"2024-09-12T03:04:09.833566+0000","last_fullsized":"2024-09-12T03:04:09.833566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:07.108262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671925+0000","last_change":"2024-09-12T03:03:07.447753+0000","last_active":"2024-09-12T03:03:41.671925+0000","last_peered":"2024-09-12T03:03:41.671925+0000","last_clean":"2024-09-12T03:03:41.671925+0000","last_became_active":"2024-09-12T03:03:07.447364+0000","last_became_peered":"2024-09-12T03:03:07.447364+0000","last_unstale":"2024-09-12T03:03:41.671925+0000","last_undegraded":"2024-09-12T03:03:41.671925+0000","last_fullsized":"2024-09-12T03:03:41.671925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:11.120666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667317+0000","last_change":"2024-09-12T03:03:17.521454+0000","last_active":"2024-09-12T03:03:41.667317+0000","last_peered":"2024-09-12T03:03:41.667317+0000","last_clean":"2024-09-12T03:03:41.667317+0000","last_became_active":"2024-09-12T03:03:17.521053+0000","last_became_peered":"2024-09-12T03:03:17.521053+0000","last_unstale":"2024-09-12T03:03:41.667317+0000","last_undegraded":"2024-09-12T03:03:41.667317+0000","last_fullsized":"2024-09-12T03:03:41.667317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:57.088465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669038+0000","last_change":"2024-09-12T03:03:07.451140+0000","last_active":"2024-09-12T03:03:41.669038+0000","last_peered":"2024-09-12T03:03:41.669038+0000","last_clean":"2024-09-12T03:03:41.669038+0000","last_became_active":"2024-09-12T03:03:07.450775+0000","last_became_peered":"2024-09-12T03:03:07.450775+0000","last_unstale":"2024-09-12T03:03:41.669038+0000","last_undegraded":"2024-09-12T03:03:41.669038+0000","last_fullsized":"2024-09-12T03:03:41.669038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:12.131080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667313+0000","last_change":"2024-09-12T03:03:07.445602+0000","last_active":"2024-09-12T03:03:41.667313+0000","last_peered":"2024-09-12T03:03:41.667313+0000","last_clean":"2024-09-12T03:03:41.667313+0000","last_became_active":"2024-09-12T03:03:07.445210+0000","last_became_peered":"2024-09-12T03:03:07.445210+0000","last_unstale":"2024-09-12T03:03:41.667313+0000","last_undegraded":"2024-09-12T03:03:41.667313+0000","last_fullsized":"2024-09-12T03:03:41.667313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:48.044322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667017+0000","last_change":"2024-09-12T03:03:07.453006+0000","last_active":"2024-09-12T03:03:41.667017+0000","last_peered":"2024-09-12T03:03:41.667017+0000","last_clean":"2024-09-12T03:03:41.667017+0000","last_became_active":"2024-09-12T03:03:07.452608+0000","last_became_peered":"2024-09-12T03:03:07.452608+0000","last_unstale":"2024-09-12T03:03:41.667017+0000","last_undegraded":"2024-09-12T03:03:41.667017+0000","last_fullsized":"2024-09-12T03:03:41.667017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:22.443868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670588+0000","last_change":"2024-09-12T03:03:07.457284+0000","last_active":"2024-09-12T03:03:41.670588+0000","last_peered":"2024-09-12T03:03:41.670588+0000","last_clean":"2024-09-12T03:03:41.670588+0000","last_became_active":"2024-09-12T03:03:07.456964+0000","last_became_peered":"2024-09-12T03:03:07.456964+0000","last_unstale":"2024-09-12T03:03:41.670588+0000","last_undegraded":"2024-09-12T03:03:41.670588+0000","last_fullsized":"2024-09-12T03:03:41.670588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.525035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953608+0000","last_change":"2024-09-12T03:03:07.451622+0000","last_active":"2024-09-12T03:03:41.953608+0000","last_peered":"2024-09-12T03:03:41.953608+0000","last_clean":"2024-09-12T03:03:41.953608+0000","last_became_active":"2024-09-12T03:03:07.451240+0000","last_became_peered":"2024-09-12T03:03:07.451240+0000","last_unstale":"2024-09-12T03:03:41.953608+0000","last_undegraded":"2024-09-12T03:03:41.953608+0000","last_fullsized":"2024-09-12T03:03:41.953608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:17.357741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669486+0000","last_change":"2024-09-12T03:03:07.450627+0000","last_active":"2024-09-12T03:03:41.669486+0000","last_peered":"2024-09-12T03:03:41.669486+0000","last_clean":"2024-09-12T03:03:41.669486+0000","last_became_active":"2024-09-12T03:03:07.450279+0000","last_became_peered":"2024-09-12T03:03:07.450279+0000","last_unstale":"2024-09-12T03:03:41.669486+0000","last_undegraded":"2024-09-12T03:03:41.669486+0000","last_fullsized":"2024-09-12T03:03:41.669486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:32.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667829+0000","last_change":"2024-09-12T03:03:07.444125+0000","last_active":"2024-09-12T03:03:41.667829+0000","last_peered":"2024-09-12T03:03:41.667829+0000","last_clean":"2024-09-12T03:03:41.667829+0000","last_became_active":"2024-09-12T03:03:07.443799+0000","last_became_peered":"2024-09-12T03:03:07.443799+0000","last_unstale":"2024-09-12T03:03:41.667829+0000","last_undegraded":"2024-09-12T03:03:41.667829+0000","last_fullsized":"2024-09-12T03:03:41.667829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:49.573684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672236+0000","last_change":"2024-09-12T03:03:07.446562+0000","last_active":"2024-09-12T03:03:41.672236+0000","last_peered":"2024-09-12T03:03:41.672236+0000","last_clean":"2024-09-12T03:03:41.672236+0000","last_became_active":"2024-09-12T03:03:07.445539+0000","last_became_peered":"2024-09-12T03:03:07.445539+0000","last_unstale":"2024-09-12T03:03:41.672236+0000","last_undegraded":"2024-09-12T03:03:41.672236+0000","last_fullsized":"2024-09-12T03:03:41.672236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:06.939290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454554+0000","last_change":"2024-09-12T03:03:07.444899+0000","last_active":"2024-09-12T03:03:42.454554+0000","last_peered":"2024-09-12T03:03:42.454554+0000","last_clean":"2024-09-12T03:03:42.454554+0000","last_became_active":"2024-09-12T03:03:07.443309+0000","last_became_peered":"2024-09-12T03:03:07.443309+0000","last_unstale":"2024-09-12T03:03:42.454554+0000","last_undegraded":"2024-09-12T03:03:42.454554+0000","last_fullsized":"2024-09-12T03:03:42.454554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:06.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671496+0000","last_change":"2024-09-12T03:03:07.455857+0000","last_active":"2024-09-12T03:03:41.671496+0000","last_peered":"2024-09-12T03:03:41.671496+0000","last_clean":"2024-09-12T03:03:41.671496+0000","last_became_active":"2024-09-12T03:03:07.450237+0000","last_became_peered":"2024-09-12T03:03:07.450237+0000","last_unstale":"2024-09-12T03:03:41.671496+0000","last_undegraded":"2024-09-12T03:03:41.671496+0000","last_fullsized":"2024-09-12T03:03:41.671496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:56.338030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672505+0000","last_change":"2024-09-12T03:03:07.445924+0000","last_active":"2024-09-12T03:03:41.672505+0000","last_peered":"2024-09-12T03:03:41.672505+0000","last_clean":"2024-09-12T03:03:41.672505+0000","last_became_active":"2024-09-12T03:03:07.445449+0000","last_became_peered":"2024-09-12T03:03:07.445449+0000","last_unstale":"2024-09-12T03:03:41.672505+0000","last_undegraded":"2024-09-12T03:03:41.672505+0000","last_fullsized":"2024-09-12T03:03:41.672505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:42.040044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953525+0000","last_change":"2024-09-12T03:03:07.449653+0000","last_active":"2024-09-12T03:03:41.953525+0000","last_peered":"2024-09-12T03:03:41.953525+0000","last_clean":"2024-09-12T03:03:41.953525+0000","last_became_active":"2024-09-12T03:03:07.445691+0000","last_became_peered":"2024-09-12T03:03:07.445691+0000","last_unstale":"2024-09-12T03:03:41.953525+0000","last_undegraded":"2024-09-12T03:03:41.953525+0000","last_fullsized":"2024-09-12T03:03:41.953525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:53.210238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669233+0000","last_change":"2024-09-12T03:03:07.453139+0000","last_active":"2024-09-12T03:03:41.669233+0000","last_peered":"2024-09-12T03:03:41.669233+0000","last_clean":"2024-09-12T03:03:41.669233+0000","last_became_active":"2024-09-12T03:03:07.452770+0000","last_became_peered":"2024-09-12T03:03:07.452770+0000","last_unstale":"2024-09-12T03:03:41.669233+0000","last_undegraded":"2024-09-12T03:03:41.669233+0000","last_fullsized":"2024-09-12T03:03:41.669233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:07.247283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667561+0000","last_change":"2024-09-12T03:03:07.441645+0000","last_active":"2024-09-12T03:03:41.667561+0000","last_peered":"2024-09-12T03:03:41.667561+0000","last_clean":"2024-09-12T03:03:41.667561+0000","last_became_active":"2024-09-12T03:03:07.441191+0000","last_became_peered":"2024-09-12T03:03:07.441191+0000","last_unstale":"2024-09-12T03:03:41.667561+0000","last_undegraded":"2024-09-12T03:03:41.667561+0000","last_fullsized":"2024-09-12T03:03:41.667561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:40.873214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175893+0000","last_change":"2024-09-12T03:03:07.446707+0000","last_active":"2024-09-12T03:03:42.175893+0000","last_peered":"2024-09-12T03:03:42.175893+0000","last_clean":"2024-09-12T03:03:42.175893+0000","last_became_active":"2024-09-12T03:03:07.446301+0000","last_became_peered":"2024-09-12T03:03:07.446301+0000","last_unstale":"2024-09-12T03:03:42.175893+0000","last_undegraded":"2024-09-12T03:03:42.175893+0000","last_fullsized":"2024-09-12T03:03:42.175893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:30.610892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669605+0000","last_change":"2024-09-12T03:03:07.449478+0000","last_active":"2024-09-12T03:03:41.669605+0000","last_peered":"2024-09-12T03:03:41.669605+0000","last_clean":"2024-09-12T03:03:41.669605+0000","last_became_active":"2024-09-12T03:03:07.448830+0000","last_became_peered":"2024-09-12T03:03:07.448830+0000","last_unstale":"2024-09-12T03:03:41.669605+0000","last_undegraded":"2024-09-12T03:03:41.669605+0000","last_fullsized":"2024-09-12T03:03:41.669605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:55.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670382+0000","last_change":"2024-09-12T03:03:07.449866+0000","last_active":"2024-09-12T03:03:41.670382+0000","last_peered":"2024-09-12T03:03:41.670382+0000","last_clean":"2024-09-12T03:03:41.670382+0000","last_became_active":"2024-09-12T03:03:07.448934+0000","last_became_peered":"2024-09-12T03:03:07.448934+0000","last_unstale":"2024-09-12T03:03:41.670382+0000","last_undegraded":"2024-09-12T03:03:41.670382+0000","last_fullsized":"2024-09-12T03:03:41.670382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:24.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673404+0000","last_change":"2024-09-12T03:03:41.673404+0000","last_active":"2024-09-12T03:03:41.673404+0000","last_peered":"2024-09-12T03:03:41.673404+0000","last_clean":"2024-09-12T03:03:41.673404+0000","last_became_active":"2024-09-12T03:03:41.673022+0000","last_became_peered":"2024-09-12T03:03:41.673022+0000","last_unstale":"2024-09-12T03:03:41.673404+0000","last_undegraded":"2024-09-12T03:03:41.673404+0000","last_fullsized":"2024-09-12T03:03:41.673404+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:44.083090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669757+0000","last_change":"2024-09-12T03:03:07.455058+0000","last_active":"2024-09-12T03:03:41.669757+0000","last_peered":"2024-09-12T03:03:41.669757+0000","last_clean":"2024-09-12T03:03:41.669757+0000","last_became_active":"2024-09-12T03:03:07.449331+0000","last_became_peered":"2024-09-12T03:03:07.449331+0000","last_unstale":"2024-09-12T03:03:41.669757+0000","last_undegraded":"2024-09-12T03:03:41.669757+0000","last_fullsized":"2024-09-12T03:03:41.669757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:30.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953519+0000","last_change":"2024-09-12T03:03:07.447193+0000","last_active":"2024-09-12T03:03:41.953519+0000","last_peered":"2024-09-12T03:03:41.953519+0000","last_clean":"2024-09-12T03:03:41.953519+0000","last_became_active":"2024-09-12T03:03:07.445206+0000","last_became_peered":"2024-09-12T03:03:07.445206+0000","last_unstale":"2024-09-12T03:03:41.953519+0000","last_undegraded":"2024-09-12T03:03:41.953519+0000","last_fullsized":"2024-09-12T03:03:41.953519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:28.919293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.674184+0000","last_change":"2024-09-12T03:03:07.442621+0000","last_active":"2024-09-12T03:03:41.674184+0000","last_peered":"2024-09-12T03:03:41.674184+0000","last_clean":"2024-09-12T03:03:41.674184+0000","last_became_active":"2024-09-12T03:03:07.441124+0000","last_became_peered":"2024-09-12T03:03:07.441124+0000","last_unstale":"2024-09-12T03:03:41.674184+0000","last_undegraded":"2024-09-12T03:03:41.674184+0000","last_fullsized":"2024-09-12T03:03:41.674184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:08.442627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672123+0000","last_change":"2024-09-12T03:03:07.443619+0000","last_active":"2024-09-12T03:03:41.672123+0000","last_peered":"2024-09-12T03:03:41.672123+0000","last_clean":"2024-09-12T03:03:41.672123+0000","last_became_active":"2024-09-12T03:03:07.443290+0000","last_became_peered":"2024-09-12T03:03:07.443290+0000","last_unstale":"2024-09-12T03:03:41.672123+0000","last_undegraded":"2024-09-12T03:03:41.672123+0000","last_fullsized":"2024-09-12T03:03:41.672123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:43.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.877152+0000","last_change":"2024-09-12T03:03:07.454164+0000","last_active":"2024-09-12T03:03:41.877152+0000","last_peered":"2024-09-12T03:03:41.877152+0000","last_clean":"2024-09-12T03:03:41.877152+0000","last_became_active":"2024-09-12T03:03:07.453835+0000","last_became_peered":"2024-09-12T03:03:07.453835+0000","last_unstale":"2024-09-12T03:03:41.877152+0000","last_undegraded":"2024-09-12T03:03:41.877152+0000","last_fullsized":"2024-09-12T03:03:41.877152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:00.494720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672224+0000","last_change":"2024-09-12T03:03:07.446854+0000","last_active":"2024-09-12T03:03:41.672224+0000","last_peered":"2024-09-12T03:03:41.672224+0000","last_clean":"2024-09-12T03:03:41.672224+0000","last_became_active":"2024-09-12T03:03:07.446411+0000","last_became_peered":"2024-09-12T03:03:07.446411+0000","last_unstale":"2024-09-12T03:03:41.672224+0000","last_undegraded":"2024-09-12T03:03:41.672224+0000","last_fullsized":"2024-09-12T03:03:41.672224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:54.917686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671209+0000","last_change":"2024-09-12T03:03:07.448332+0000","last_active":"2024-09-12T03:03:41.671209+0000","last_peered":"2024-09-12T03:03:41.671209+0000","last_clean":"2024-09-12T03:03:41.671209+0000","last_became_active":"2024-09-12T03:03:07.447776+0000","last_became_peered":"2024-09-12T03:03:07.447776+0000","last_unstale":"2024-09-12T03:03:41.671209+0000","last_undegraded":"2024-09-12T03:03:41.671209+0000","last_fullsized":"2024-09-12T03:03:41.671209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:08.531088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671147+0000","last_change":"2024-09-12T03:03:07.449781+0000","last_active":"2024-09-12T03:03:41.671147+0000","last_peered":"2024-09-12T03:03:41.671147+0000","last_clean":"2024-09-12T03:03:41.671147+0000","last_became_active":"2024-09-12T03:03:07.449359+0000","last_became_peered":"2024-09-12T03:03:07.449359+0000","last_unstale":"2024-09-12T03:03:41.671147+0000","last_undegraded":"2024-09-12T03:03:41.671147+0000","last_fullsized":"2024-09-12T03:03:41.671147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:11.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.877017+0000","last_change":"2024-09-12T03:03:07.453881+0000","last_active":"2024-09-12T03:03:41.877017+0000","last_peered":"2024-09-12T03:03:41.877017+0000","last_clean":"2024-09-12T03:03:41.877017+0000","last_became_active":"2024-09-12T03:03:07.453455+0000","last_became_peered":"2024-09-12T03:03:07.453455+0000","last_unstale":"2024-09-12T03:03:41.877017+0000","last_undegraded":"2024-09-12T03:03:41.877017+0000","last_fullsized":"2024-09-12T03:03:41.877017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:18:46.797598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671873+0000","last_change":"2024-09-12T03:03:07.459332+0000","last_active":"2024-09-12T03:03:41.671873+0000","last_peered":"2024-09-12T03:03:41.671873+0000","last_clean":"2024-09-12T03:03:41.671873+0000","last_became_active":"2024-09-12T03:03:07.459032+0000","last_became_peered":"2024-09-12T03:03:07.459032+0000","last_unstale":"2024-09-12T03:03:41.671873+0000","last_undegraded":"2024-09-12T03:03:41.671873+0000","last_fullsized":"2024-09-12T03:03:41.671873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:16.702782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.674083+0000","last_change":"2024-09-12T03:03:07.441696+0000","last_active":"2024-09-12T03:03:41.674083+0000","last_peered":"2024-09-12T03:03:41.674083+0000","last_clean":"2024-09-12T03:03:41.674083+0000","last_became_active":"2024-09-12T03:03:07.440575+0000","last_became_peered":"2024-09-12T03:03:07.440575+0000","last_unstale":"2024-09-12T03:03:41.674083+0000","last_undegraded":"2024-09-12T03:03:41.674083+0000","last_fullsized":"2024-09-12T03:03:41.674083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:26.049893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669856+0000","last_change":"2024-09-12T03:03:07.454945+0000","last_active":"2024-09-12T03:03:41.669856+0000","last_peered":"2024-09-12T03:03:41.669856+0000","last_clean":"2024-09-12T03:03:41.669856+0000","last_became_active":"2024-09-12T03:03:07.452368+0000","last_became_peered":"2024-09-12T03:03:07.452368+0000","last_unstale":"2024-09-12T03:03:41.669856+0000","last_undegraded":"2024-09-12T03:03:41.669856+0000","last_fullsized":"2024-09-12T03:03:41.669856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:21.169859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669980+0000","last_change":"2024-09-12T03:03:07.454230+0000","last_active":"2024-09-12T03:03:41.669980+0000","last_peered":"2024-09-12T03:03:41.669980+0000","last_clean":"2024-09-12T03:03:41.669980+0000","last_became_active":"2024-09-12T03:03:07.453913+0000","last_became_peered":"2024-09-12T03:03:07.453913+0000","last_unstale":"2024-09-12T03:03:41.669980+0000","last_undegraded":"2024-09-12T03:03:41.669980+0000","last_fullsized":"2024-09-12T03:03:41.669980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:56.264048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669558+0000","last_change":"2024-09-12T03:03:07.449227+0000","last_active":"2024-09-12T03:03:41.669558+0000","last_peered":"2024-09-12T03:03:41.669558+0000","last_clean":"2024-09-12T03:03:41.669558+0000","last_became_active":"2024-09-12T03:03:07.448605+0000","last_became_peered":"2024-09-12T03:03:07.448605+0000","last_unstale":"2024-09-12T03:03:41.669558+0000","last_undegraded":"2024-09-12T03:03:41.669558+0000","last_fullsized":"2024-09-12T03:03:41.669558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:06.420752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61261,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":129,"num_read_kb":131,"num_write":160,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":171008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481819,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":535416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32900,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712252,"statfs":{"total":95995035648,"available":95961346048,"internally_reserved":0,"allocated":1757184,"data_stored":1179942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":539748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135915,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":535572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1131483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":530828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1127071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":535478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":531022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":576964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":587996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":532216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":46226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:04:17.204 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:04:17.592 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:17 smithi057 ceph-mon[30361]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:04:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:17 smithi057 ceph-mon[30361]: from='client.24767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:17 smithi204 ceph-mon[35013]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:04:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:17 smithi204 ceph-mon[35013]: from='client.24767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:17 smithi181 ceph-mon[35320]: pgmap v342: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-12T03:04:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:17 smithi181 ceph-mon[35320]: from='client.24767 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 -- 172.21.15.57:0/1912991526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c103ff0 msgr2=0x7f661c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 --2- 172.21.15.57:0/1912991526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c103ff0 0x7f661c1043d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f660c009900 tx=0x7f660c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 -- 172.21.15.57:0/1912991526 shutdown_connections 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 --2- 172.21.15.57:0/1912991526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 0x7f661c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 --2- 172.21.15.57:0/1912991526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c1049a0 0x7f661c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 --2- 172.21.15.57:0/1912991526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c103ff0 0x7f661c1043d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.139 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 -- 172.21.15.57:0/1912991526 >> 172.21.15.57:0/1912991526 conn(0x7f661c0ffa90 msgr2=0x7f661c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 -- 172.21.15.57:0/1912991526 shutdown_connections 2024-09-12T03:04:18.140 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.138+0000 7f6620a00640 1 -- 172.21.15.57:0/1912991526 wait complete. 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.139+0000 7f6620a00640 1 Processor -- start 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.139+0000 7f6620a00640 1 -- start start 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c103ff0 0x7f661c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 0x7f661c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f661c114120 con 0x7f661c1049a0 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f661c113fa0 con 0x7f661c109120 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6620a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f661c1142a0 con 0x7f661c103ff0 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:18.142 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c103ff0 0x7f661c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47944/0 (socket says 172.21.15.57:47944) 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 -- 172.21.15.57:0/2250433458 learned_addr learned my addr 172.21.15.57:0/2250433458 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f661a800640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 0x7f661c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 -- 172.21.15.57:0/2250433458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c103ff0 msgr2=0x7f661c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c103ff0 0x7f661c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 -- 172.21.15.57:0/2250433458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 msgr2=0x7f661c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 0x7f661c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.140+0000 7f6619400640 1 -- 172.21.15.57:0/2250433458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661c19f590 con 0x7f661c1049a0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6619400640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f660800e9f0 tx=0x7f660800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660800cd80 con 0x7f661c1049a0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6608010910 con 0x7f661c1049a0 2024-09-12T03:04:18.143 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661c19f880 con 0x7f661c1049a0 2024-09-12T03:04:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608018990 con 0x7f661c1049a0 2024-09-12T03:04:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.141+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661c1a7160 con 0x7f661c1049a0 2024-09-12T03:04:18.144 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.143+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661c1043d0 con 0x7f661c1049a0 2024-09-12T03:04:18.145 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.145+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6608010ac0 con 0x7f661c1049a0 2024-09-12T03:04:18.146 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.146+0000 7f6612a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.146+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f66080cf8a0 con 0x7f661c1049a0 2024-09-12T03:04:18.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.147+0000 7f6619e00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:18.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.147+0000 7f6619e00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 0x7f66000a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f660c009230 tx=0x7f660c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:18.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.151+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f660809bc90 con 0x7f661c1049a0 2024-09-12T03:04:18.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.332+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f661c062630 con 0x7f66000a2c10 2024-09-12T03:04:18.344 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.343+0000 7f6612a00640 1 -- 172.21.15.57:0/2250433458 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7f661c062630 con 0x7f66000a2c10 2024-09-12T03:04:18.350 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:18.352 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 msgr2=0x7f66000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 0x7f66000a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f660c009230 tx=0x7f660c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 msgr2=0x7f661c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f660800e9f0 tx=0x7f660800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 shutdown_connections 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66000a2c10 0x7f66000a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661c109120 0x7f661c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f661c1049a0 0x7f661c19ab20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 --2- 172.21.15.57:0/2250433458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f661c103ff0 0x7f661c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.348+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 >> 172.21.15.57:0/2250433458 conn(0x7f661c0ffa90 msgr2=0x7f661c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:18.355 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.349+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 shutdown_connections 2024-09-12T03:04:18.356 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:18.349+0000 7f6620a00640 1 -- 172.21.15.57:0/2250433458 wait complete. 2024-09-12T03:04:19.308 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-12T03:04:17.727757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651863,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":280,"num_read_kb":255,"num_write":343,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372216,"kb_used_data":14648,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569608,"statfs":{"total":1151940427776,"available":1151559278592,"internally_reserved":0,"allocated":14999552,"data_stored":8356511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33305,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":69,"num_write":23,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003733"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809294+0000","last_change":"2024-09-12T03:03:04.425029+0000","last_active":"2024-09-12T03:03:55.809294+0000","last_peered":"2024-09-12T03:03:55.809294+0000","last_clean":"2024-09-12T03:03:55.809294+0000","last_became_active":"2024-09-12T03:03:04.423326+0000","last_became_peered":"2024-09-12T03:03:04.423326+0000","last_unstale":"2024-09-12T03:03:55.809294+0000","last_undegraded":"2024-09-12T03:03:55.809294+0000","last_fullsized":"2024-09-12T03:03:55.809294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:28.269107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672403+0000","last_change":"2024-09-12T03:03:07.449348+0000","last_active":"2024-09-12T03:03:41.672403+0000","last_peered":"2024-09-12T03:03:41.672403+0000","last_clean":"2024-09-12T03:03:41.672403+0000","last_became_active":"2024-09-12T03:03:07.448813+0000","last_became_peered":"2024-09-12T03:03:07.448813+0000","last_unstale":"2024-09-12T03:03:41.672403+0000","last_undegraded":"2024-09-12T03:03:41.672403+0000","last_fullsized":"2024-09-12T03:03:41.672403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:31.404583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.666957+0000","last_change":"2024-09-12T03:03:17.530646+0000","last_active":"2024-09-12T03:03:41.666957+0000","last_peered":"2024-09-12T03:03:41.666957+0000","last_clean":"2024-09-12T03:03:41.666957+0000","last_became_active":"2024-09-12T03:03:17.530228+0000","last_became_peered":"2024-09-12T03:03:17.530228+0000","last_unstale":"2024-09-12T03:03:41.666957+0000","last_undegraded":"2024-09-12T03:03:41.666957+0000","last_fullsized":"2024-09-12T03:03:41.666957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.763496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672344+0000","last_change":"2024-09-12T03:03:04.415539+0000","last_active":"2024-09-12T03:03:41.672344+0000","last_peered":"2024-09-12T03:03:41.672344+0000","last_clean":"2024-09-12T03:03:41.672344+0000","last_became_active":"2024-09-12T03:03:04.415163+0000","last_became_peered":"2024-09-12T03:03:04.415163+0000","last_unstale":"2024-09-12T03:03:41.672344+0000","last_undegraded":"2024-09-12T03:03:41.672344+0000","last_fullsized":"2024-09-12T03:03:41.672344+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:12.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670669+0000","last_change":"2024-09-12T03:03:07.447808+0000","last_active":"2024-09-12T03:03:41.670669+0000","last_peered":"2024-09-12T03:03:41.670669+0000","last_clean":"2024-09-12T03:03:41.670669+0000","last_became_active":"2024-09-12T03:03:07.446760+0000","last_became_peered":"2024-09-12T03:03:07.446760+0000","last_unstale":"2024-09-12T03:03:41.670669+0000","last_undegraded":"2024-09-12T03:03:41.670669+0000","last_fullsized":"2024-09-12T03:03:41.670669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:56.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671481+0000","last_change":"2024-09-12T03:03:17.530408+0000","last_active":"2024-09-12T03:03:41.671481+0000","last_peered":"2024-09-12T03:03:41.671481+0000","last_clean":"2024-09-12T03:03:41.671481+0000","last_became_active":"2024-09-12T03:03:17.530073+0000","last_became_peered":"2024-09-12T03:03:17.530073+0000","last_unstale":"2024-09-12T03:03:41.671481+0000","last_undegraded":"2024-09-12T03:03:41.671481+0000","last_fullsized":"2024-09-12T03:03:41.671481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:57.199786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807592+0000","last_change":"2024-09-12T03:03:04.422168+0000","last_active":"2024-09-12T03:03:55.807592+0000","last_peered":"2024-09-12T03:03:55.807592+0000","last_clean":"2024-09-12T03:03:55.807592+0000","last_became_active":"2024-09-12T03:03:04.421739+0000","last_became_peered":"2024-09-12T03:03:04.421739+0000","last_unstale":"2024-09-12T03:03:55.807592+0000","last_undegraded":"2024-09-12T03:03:55.807592+0000","last_fullsized":"2024-09-12T03:03:55.807592+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:28.080796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672110+0000","last_change":"2024-09-12T03:03:07.445256+0000","last_active":"2024-09-12T03:03:41.672110+0000","last_peered":"2024-09-12T03:03:41.672110+0000","last_clean":"2024-09-12T03:03:41.672110+0000","last_became_active":"2024-09-12T03:03:07.444919+0000","last_became_peered":"2024-09-12T03:03:07.444919+0000","last_unstale":"2024-09-12T03:03:41.672110+0000","last_undegraded":"2024-09-12T03:03:41.672110+0000","last_fullsized":"2024-09-12T03:03:41.672110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:42.826440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456763+0000","last_change":"2024-09-12T03:03:42.456763+0000","last_active":"2024-09-12T03:03:42.456763+0000","last_peered":"2024-09-12T03:03:42.456763+0000","last_clean":"2024-09-12T03:03:42.456763+0000","last_became_active":"2024-09-12T03:03:42.456267+0000","last_became_peered":"2024-09-12T03:03:42.456267+0000","last_unstale":"2024-09-12T03:03:42.456763+0000","last_undegraded":"2024-09-12T03:03:42.456763+0000","last_fullsized":"2024-09-12T03:03:42.456763+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:17.092878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672323+0000","last_change":"2024-09-12T03:03:04.423817+0000","last_active":"2024-09-12T03:03:41.672323+0000","last_peered":"2024-09-12T03:03:41.672323+0000","last_clean":"2024-09-12T03:03:41.672323+0000","last_became_active":"2024-09-12T03:03:04.422617+0000","last_became_peered":"2024-09-12T03:03:04.422617+0000","last_unstale":"2024-09-12T03:03:41.672323+0000","last_undegraded":"2024-09-12T03:03:41.672323+0000","last_fullsized":"2024-09-12T03:03:41.672323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.309338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454778+0000","last_change":"2024-09-12T03:03:07.454768+0000","last_active":"2024-09-12T03:03:42.454778+0000","last_peered":"2024-09-12T03:03:42.454778+0000","last_clean":"2024-09-12T03:03:42.454778+0000","last_became_active":"2024-09-12T03:03:07.454414+0000","last_became_peered":"2024-09-12T03:03:07.454414+0000","last_unstale":"2024-09-12T03:03:42.454778+0000","last_undegraded":"2024-09-12T03:03:42.454778+0000","last_fullsized":"2024-09-12T03:03:42.454778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:39.968938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671409+0000","last_change":"2024-09-12T03:03:17.524984+0000","last_active":"2024-09-12T03:03:41.671409+0000","last_peered":"2024-09-12T03:03:41.671409+0000","last_clean":"2024-09-12T03:03:41.671409+0000","last_became_active":"2024-09-12T03:03:17.524516+0000","last_became_peered":"2024-09-12T03:03:17.524516+0000","last_unstale":"2024-09-12T03:03:41.671409+0000","last_undegraded":"2024-09-12T03:03:41.671409+0000","last_fullsized":"2024-09-12T03:03:41.671409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:34.342631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671167+0000","last_change":"2024-09-12T03:03:07.450348+0000","last_active":"2024-09-12T03:03:41.671167+0000","last_peered":"2024-09-12T03:03:41.671167+0000","last_clean":"2024-09-12T03:03:41.671167+0000","last_became_active":"2024-09-12T03:03:07.448968+0000","last_became_peered":"2024-09-12T03:03:07.448968+0000","last_unstale":"2024-09-12T03:03:41.671167+0000","last_undegraded":"2024-09-12T03:03:41.671167+0000","last_fullsized":"2024-09-12T03:03:41.671167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:19.053409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798384+0000","last_change":"2024-09-12T03:03:41.676526+0000","last_active":"2024-09-12T03:03:54.798384+0000","last_peered":"2024-09-12T03:03:54.798384+0000","last_clean":"2024-09-12T03:03:54.798384+0000","last_became_active":"2024-09-12T03:03:41.673301+0000","last_became_peered":"2024-09-12T03:03:41.673301+0000","last_unstale":"2024-09-12T03:03:54.798384+0000","last_undegraded":"2024-09-12T03:03:54.798384+0000","last_fullsized":"2024-09-12T03:03:54.798384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:04.648668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673644+0000","last_change":"2024-09-12T03:03:17.526846+0000","last_active":"2024-09-12T03:03:41.673644+0000","last_peered":"2024-09-12T03:03:41.673644+0000","last_clean":"2024-09-12T03:03:41.673644+0000","last_became_active":"2024-09-12T03:03:17.526412+0000","last_became_peered":"2024-09-12T03:03:17.526412+0000","last_unstale":"2024-09-12T03:03:41.673644+0000","last_undegraded":"2024-09-12T03:03:41.673644+0000","last_fullsized":"2024-09-12T03:03:41.673644+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671567+0000","last_change":"2024-09-12T03:03:07.453989+0000","last_active":"2024-09-12T03:03:41.671567+0000","last_peered":"2024-09-12T03:03:41.671567+0000","last_clean":"2024-09-12T03:03:41.671567+0000","last_became_active":"2024-09-12T03:03:07.451796+0000","last_became_peered":"2024-09-12T03:03:07.451796+0000","last_unstale":"2024-09-12T03:03:41.671567+0000","last_undegraded":"2024-09-12T03:03:41.671567+0000","last_fullsized":"2024-09-12T03:03:41.671567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:27.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667160+0000","last_change":"2024-09-12T03:03:04.423756+0000","last_active":"2024-09-12T03:03:41.667160+0000","last_peered":"2024-09-12T03:03:41.667160+0000","last_clean":"2024-09-12T03:03:41.667160+0000","last_became_active":"2024-09-12T03:03:04.423075+0000","last_became_peered":"2024-09-12T03:03:04.423075+0000","last_unstale":"2024-09-12T03:03:41.667160+0000","last_undegraded":"2024-09-12T03:03:41.667160+0000","last_fullsized":"2024-09-12T03:03:41.667160+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.201739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673954+0000","last_change":"2024-09-12T03:03:17.532906+0000","last_active":"2024-09-12T03:03:41.673954+0000","last_peered":"2024-09-12T03:03:41.673954+0000","last_clean":"2024-09-12T03:03:41.673954+0000","last_became_active":"2024-09-12T03:03:17.532343+0000","last_became_peered":"2024-09-12T03:03:17.532343+0000","last_unstale":"2024-09-12T03:03:41.673954+0000","last_undegraded":"2024-09-12T03:03:41.673954+0000","last_fullsized":"2024-09-12T03:03:41.673954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:05.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669111+0000","last_change":"2024-09-12T03:03:07.450375+0000","last_active":"2024-09-12T03:03:41.669111+0000","last_peered":"2024-09-12T03:03:41.669111+0000","last_clean":"2024-09-12T03:03:41.669111+0000","last_became_active":"2024-09-12T03:03:07.450061+0000","last_became_peered":"2024-09-12T03:03:07.450061+0000","last_unstale":"2024-09-12T03:03:41.669111+0000","last_undegraded":"2024-09-12T03:03:41.669111+0000","last_fullsized":"2024-09-12T03:03:41.669111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:05.922237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798743+0000","last_change":"2024-09-12T03:03:04.423664+0000","last_active":"2024-09-12T03:03:54.798743+0000","last_peered":"2024-09-12T03:03:54.798743+0000","last_clean":"2024-09-12T03:03:54.798743+0000","last_became_active":"2024-09-12T03:03:04.423285+0000","last_became_peered":"2024-09-12T03:03:04.423285+0000","last_unstale":"2024-09-12T03:03:54.798743+0000","last_undegraded":"2024-09-12T03:03:54.798743+0000","last_fullsized":"2024-09-12T03:03:54.798743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.782484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175832+0000","last_change":"2024-09-12T03:03:17.535385+0000","last_active":"2024-09-12T03:03:42.175832+0000","last_peered":"2024-09-12T03:03:42.175832+0000","last_clean":"2024-09-12T03:03:42.175832+0000","last_became_active":"2024-09-12T03:03:17.534775+0000","last_became_peered":"2024-09-12T03:03:17.534775+0000","last_unstale":"2024-09-12T03:03:42.175832+0000","last_undegraded":"2024-09-12T03:03:42.175832+0000","last_fullsized":"2024-09-12T03:03:42.175832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:02.549462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671807+0000","last_change":"2024-09-12T03:03:07.442786+0000","last_active":"2024-09-12T03:03:41.671807+0000","last_peered":"2024-09-12T03:03:41.671807+0000","last_clean":"2024-09-12T03:03:41.671807+0000","last_became_active":"2024-09-12T03:03:07.442456+0000","last_became_peered":"2024-09-12T03:03:07.442456+0000","last_unstale":"2024-09-12T03:03:41.671807+0000","last_undegraded":"2024-09-12T03:03:41.671807+0000","last_fullsized":"2024-09-12T03:03:41.671807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.882361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.799012+0000","last_change":"2024-09-12T03:03:41.675913+0000","last_active":"2024-09-12T03:03:54.799012+0000","last_peered":"2024-09-12T03:03:54.799012+0000","last_clean":"2024-09-12T03:03:54.799012+0000","last_became_active":"2024-09-12T03:03:41.675434+0000","last_became_peered":"2024-09-12T03:03:41.675434+0000","last_unstale":"2024-09-12T03:03:54.799012+0000","last_undegraded":"2024-09-12T03:03:54.799012+0000","last_fullsized":"2024-09-12T03:03:54.799012+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:36.484404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.177089+0000","last_change":"2024-09-12T03:03:42.177089+0000","last_active":"2024-09-12T03:03:42.177089+0000","last_peered":"2024-09-12T03:03:42.177089+0000","last_clean":"2024-09-12T03:03:42.177089+0000","last_became_active":"2024-09-12T03:03:42.176602+0000","last_became_peered":"2024-09-12T03:03:42.176602+0000","last_unstale":"2024-09-12T03:03:42.177089+0000","last_undegraded":"2024-09-12T03:03:42.177089+0000","last_fullsized":"2024-09-12T03:03:42.177089+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:04.336826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670074+0000","last_change":"2024-09-12T03:03:07.457972+0000","last_active":"2024-09-12T03:03:41.670074+0000","last_peered":"2024-09-12T03:03:41.670074+0000","last_clean":"2024-09-12T03:03:41.670074+0000","last_became_active":"2024-09-12T03:03:07.457681+0000","last_became_peered":"2024-09-12T03:03:07.457681+0000","last_unstale":"2024-09-12T03:03:41.670074+0000","last_undegraded":"2024-09-12T03:03:41.670074+0000","last_fullsized":"2024-09-12T03:03:41.670074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:26.733432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.797099+0000","last_change":"2024-09-12T03:03:04.629721+0000","last_active":"2024-09-12T03:03:54.797099+0000","last_peered":"2024-09-12T03:03:54.797099+0000","last_clean":"2024-09-12T03:03:54.797099+0000","last_became_active":"2024-09-12T03:03:04.629362+0000","last_became_peered":"2024-09-12T03:03:04.629362+0000","last_unstale":"2024-09-12T03:03:54.797099+0000","last_undegraded":"2024-09-12T03:03:54.797099+0000","last_fullsized":"2024-09-12T03:03:54.797099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:09.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667128+0000","last_change":"2024-09-12T03:03:17.521048+0000","last_active":"2024-09-12T03:03:41.667128+0000","last_peered":"2024-09-12T03:03:41.667128+0000","last_clean":"2024-09-12T03:03:41.667128+0000","last_became_active":"2024-09-12T03:03:17.520640+0000","last_became_peered":"2024-09-12T03:03:17.520640+0000","last_unstale":"2024-09-12T03:03:41.667128+0000","last_undegraded":"2024-09-12T03:03:41.667128+0000","last_fullsized":"2024-09-12T03:03:41.667128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:06.243255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672329+0000","last_change":"2024-09-12T03:03:07.445497+0000","last_active":"2024-09-12T03:03:41.672329+0000","last_peered":"2024-09-12T03:03:41.672329+0000","last_clean":"2024-09-12T03:03:41.672329+0000","last_became_active":"2024-09-12T03:03:07.442245+0000","last_became_peered":"2024-09-12T03:03:07.442245+0000","last_unstale":"2024-09-12T03:03:41.672329+0000","last_undegraded":"2024-09-12T03:03:41.672329+0000","last_fullsized":"2024-09-12T03:03:41.672329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:13.147572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:10.464993+0000","last_change":"2024-09-12T03:03:04.424438+0000","last_active":"2024-09-12T03:04:10.464993+0000","last_peered":"2024-09-12T03:04:10.464993+0000","last_clean":"2024-09-12T03:04:10.464993+0000","last_became_active":"2024-09-12T03:03:04.424078+0000","last_became_peered":"2024-09-12T03:03:04.424078+0000","last_unstale":"2024-09-12T03:04:10.464993+0000","last_undegraded":"2024-09-12T03:04:10.464993+0000","last_fullsized":"2024-09-12T03:04:10.464993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:33.090082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671140+0000","last_change":"2024-09-12T03:03:17.532767+0000","last_active":"2024-09-12T03:03:41.671140+0000","last_peered":"2024-09-12T03:03:41.671140+0000","last_clean":"2024-09-12T03:03:41.671140+0000","last_became_active":"2024-09-12T03:03:17.532408+0000","last_became_peered":"2024-09-12T03:03:17.532408+0000","last_unstale":"2024-09-12T03:03:41.671140+0000","last_undegraded":"2024-09-12T03:03:41.671140+0000","last_fullsized":"2024-09-12T03:03:41.671140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:29.294605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953710+0000","last_change":"2024-09-12T03:03:07.448931+0000","last_active":"2024-09-12T03:03:41.953710+0000","last_peered":"2024-09-12T03:03:41.953710+0000","last_clean":"2024-09-12T03:03:41.953710+0000","last_became_active":"2024-09-12T03:03:07.447204+0000","last_became_peered":"2024-09-12T03:03:07.447204+0000","last_unstale":"2024-09-12T03:03:41.953710+0000","last_undegraded":"2024-09-12T03:03:41.953710+0000","last_fullsized":"2024-09-12T03:03:41.953710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:10.491591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.801007+0000","last_change":"2024-09-12T03:03:04.425805+0000","last_active":"2024-09-12T03:03:54.801007+0000","last_peered":"2024-09-12T03:03:54.801007+0000","last_clean":"2024-09-12T03:03:54.801007+0000","last_became_active":"2024-09-12T03:03:04.425469+0000","last_became_peered":"2024-09-12T03:03:04.425469+0000","last_unstale":"2024-09-12T03:03:54.801007+0000","last_undegraded":"2024-09-12T03:03:54.801007+0000","last_fullsized":"2024-09-12T03:03:54.801007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:27.976934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669627+0000","last_change":"2024-09-12T03:03:17.525537+0000","last_active":"2024-09-12T03:03:41.669627+0000","last_peered":"2024-09-12T03:03:41.669627+0000","last_clean":"2024-09-12T03:03:41.669627+0000","last_became_active":"2024-09-12T03:03:17.525030+0000","last_became_peered":"2024-09-12T03:03:17.525030+0000","last_unstale":"2024-09-12T03:03:41.669627+0000","last_undegraded":"2024-09-12T03:03:41.669627+0000","last_fullsized":"2024-09-12T03:03:41.669627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:53.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.876773+0000","last_change":"2024-09-12T03:03:07.443712+0000","last_active":"2024-09-12T03:03:41.876773+0000","last_peered":"2024-09-12T03:03:41.876773+0000","last_clean":"2024-09-12T03:03:41.876773+0000","last_became_active":"2024-09-12T03:03:07.443374+0000","last_became_peered":"2024-09-12T03:03:07.443374+0000","last_unstale":"2024-09-12T03:03:41.876773+0000","last_undegraded":"2024-09-12T03:03:41.876773+0000","last_fullsized":"2024-09-12T03:03:41.876773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:09.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175945+0000","last_change":"2024-09-12T03:03:04.427683+0000","last_active":"2024-09-12T03:03:42.175945+0000","last_peered":"2024-09-12T03:03:42.175945+0000","last_clean":"2024-09-12T03:03:42.175945+0000","last_became_active":"2024-09-12T03:03:04.427276+0000","last_became_peered":"2024-09-12T03:03:04.427276+0000","last_unstale":"2024-09-12T03:03:42.175945+0000","last_undegraded":"2024-09-12T03:03:42.175945+0000","last_fullsized":"2024-09-12T03:03:42.175945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:44.540369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.878110+0000","last_change":"2024-09-12T03:03:41.878110+0000","last_active":"2024-09-12T03:03:41.878110+0000","last_peered":"2024-09-12T03:03:41.878110+0000","last_clean":"2024-09-12T03:03:41.878110+0000","last_became_active":"2024-09-12T03:03:41.877719+0000","last_became_peered":"2024-09-12T03:03:41.877719+0000","last_unstale":"2024-09-12T03:03:41.878110+0000","last_undegraded":"2024-09-12T03:03:41.878110+0000","last_fullsized":"2024-09-12T03:03:41.878110+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:08.312660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.876744+0000","last_change":"2024-09-12T03:03:07.454093+0000","last_active":"2024-09-12T03:03:41.876744+0000","last_peered":"2024-09-12T03:03:41.876744+0000","last_clean":"2024-09-12T03:03:41.876744+0000","last_became_active":"2024-09-12T03:03:07.453676+0000","last_became_peered":"2024-09-12T03:03:07.453676+0000","last_unstale":"2024-09-12T03:03:41.876744+0000","last_undegraded":"2024-09-12T03:03:41.876744+0000","last_fullsized":"2024-09-12T03:03:41.876744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:18.045572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671400+0000","last_change":"2024-09-12T03:03:04.425336+0000","last_active":"2024-09-12T03:03:41.671400+0000","last_peered":"2024-09-12T03:03:41.671400+0000","last_clean":"2024-09-12T03:03:41.671400+0000","last_became_active":"2024-09-12T03:03:04.424890+0000","last_became_peered":"2024-09-12T03:03:04.424890+0000","last_unstale":"2024-09-12T03:03:41.671400+0000","last_undegraded":"2024-09-12T03:03:41.671400+0000","last_fullsized":"2024-09-12T03:03:41.671400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.443627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953494+0000","last_change":"2024-09-12T03:03:17.535300+0000","last_active":"2024-09-12T03:03:41.953494+0000","last_peered":"2024-09-12T03:03:41.953494+0000","last_clean":"2024-09-12T03:03:41.953494+0000","last_became_active":"2024-09-12T03:03:17.534937+0000","last_became_peered":"2024-09-12T03:03:17.534937+0000","last_unstale":"2024-09-12T03:03:41.953494+0000","last_undegraded":"2024-09-12T03:03:41.953494+0000","last_fullsized":"2024-09-12T03:03:41.953494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:54.874973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454670+0000","last_change":"2024-09-12T03:03:07.453964+0000","last_active":"2024-09-12T03:03:42.454670+0000","last_peered":"2024-09-12T03:03:42.454670+0000","last_clean":"2024-09-12T03:03:42.454670+0000","last_became_active":"2024-09-12T03:03:07.453622+0000","last_became_peered":"2024-09-12T03:03:07.453622+0000","last_unstale":"2024-09-12T03:03:42.454670+0000","last_undegraded":"2024-09-12T03:03:42.454670+0000","last_fullsized":"2024-09-12T03:03:42.454670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:17.452519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:14.840682+0000","last_change":"2024-09-12T03:03:04.428526+0000","last_active":"2024-09-12T03:04:14.840682+0000","last_peered":"2024-09-12T03:04:14.840682+0000","last_clean":"2024-09-12T03:04:14.840682+0000","last_became_active":"2024-09-12T03:03:04.428221+0000","last_became_peered":"2024-09-12T03:03:04.428221+0000","last_unstale":"2024-09-12T03:04:14.840682+0000","last_undegraded":"2024-09-12T03:04:14.840682+0000","last_fullsized":"2024-09-12T03:04:14.840682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:50.999346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953812+0000","last_change":"2024-09-12T03:03:17.535146+0000","last_active":"2024-09-12T03:03:41.953812+0000","last_peered":"2024-09-12T03:03:41.953812+0000","last_clean":"2024-09-12T03:03:41.953812+0000","last_became_active":"2024-09-12T03:03:17.534677+0000","last_became_peered":"2024-09-12T03:03:17.534677+0000","last_unstale":"2024-09-12T03:03:41.953812+0000","last_undegraded":"2024-09-12T03:03:41.953812+0000","last_fullsized":"2024-09-12T03:03:41.953812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:09.874993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953220+0000","last_change":"2024-09-12T03:03:07.431348+0000","last_active":"2024-09-12T03:03:41.953220+0000","last_peered":"2024-09-12T03:03:41.953220+0000","last_clean":"2024-09-12T03:03:41.953220+0000","last_became_active":"2024-09-12T03:03:07.430912+0000","last_became_peered":"2024-09-12T03:03:07.430912+0000","last_unstale":"2024-09-12T03:03:41.953220+0000","last_undegraded":"2024-09-12T03:03:41.953220+0000","last_fullsized":"2024-09-12T03:03:41.953220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:27.644324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807071+0000","last_change":"2024-09-12T03:03:42.456245+0000","last_active":"2024-09-12T03:03:55.807071+0000","last_peered":"2024-09-12T03:03:55.807071+0000","last_clean":"2024-09-12T03:03:55.807071+0000","last_became_active":"2024-09-12T03:03:42.455812+0000","last_became_peered":"2024-09-12T03:03:42.455812+0000","last_unstale":"2024-09-12T03:03:55.807071+0000","last_undegraded":"2024-09-12T03:03:55.807071+0000","last_fullsized":"2024-09-12T03:03:55.807071+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454997+0000","last_change":"2024-09-12T03:03:17.527371+0000","last_active":"2024-09-12T03:03:42.454997+0000","last_peered":"2024-09-12T03:03:42.454997+0000","last_clean":"2024-09-12T03:03:42.454997+0000","last_became_active":"2024-09-12T03:03:17.526818+0000","last_became_peered":"2024-09-12T03:03:17.526818+0000","last_unstale":"2024-09-12T03:03:42.454997+0000","last_undegraded":"2024-09-12T03:03:42.454997+0000","last_fullsized":"2024-09-12T03:03:42.454997+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:49.519942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671585+0000","last_change":"2024-09-12T03:03:07.451364+0000","last_active":"2024-09-12T03:03:41.671585+0000","last_peered":"2024-09-12T03:03:41.671585+0000","last_clean":"2024-09-12T03:03:41.671585+0000","last_became_active":"2024-09-12T03:03:07.450964+0000","last_became_peered":"2024-09-12T03:03:07.450964+0000","last_unstale":"2024-09-12T03:03:41.671585+0000","last_undegraded":"2024-09-12T03:03:41.671585+0000","last_fullsized":"2024-09-12T03:03:41.671585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:56.811698+0000","last_change":"2024-09-12T03:03:04.430485+0000","last_active":"2024-09-12T03:03:56.811698+0000","last_peered":"2024-09-12T03:03:56.811698+0000","last_clean":"2024-09-12T03:03:56.811698+0000","last_became_active":"2024-09-12T03:03:04.430096+0000","last_became_peered":"2024-09-12T03:03:04.430096+0000","last_unstale":"2024-09-12T03:03:56.811698+0000","last_undegraded":"2024-09-12T03:03:56.811698+0000","last_fullsized":"2024-09-12T03:03:56.811698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:26.486223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670839+0000","last_change":"2024-09-12T03:01:48.369350+0000","last_active":"2024-09-12T03:03:41.670839+0000","last_peered":"2024-09-12T03:03:41.670839+0000","last_clean":"2024-09-12T03:03:41.670839+0000","last_became_active":"2024-09-12T03:01:48.369011+0000","last_became_peered":"2024-09-12T03:01:48.369011+0000","last_unstale":"2024-09-12T03:03:41.670839+0000","last_undegraded":"2024-09-12T03:03:41.670839+0000","last_fullsized":"2024-09-12T03:03:41.670839+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_clean_scrub_stamp":"2024-09-12T02:58:06.597228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:57.348343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672013+0000","last_change":"2024-09-12T03:03:17.541157+0000","last_active":"2024-09-12T03:03:41.672013+0000","last_peered":"2024-09-12T03:03:41.672013+0000","last_clean":"2024-09-12T03:03:41.672013+0000","last_became_active":"2024-09-12T03:03:17.540687+0000","last_became_peered":"2024-09-12T03:03:17.540687+0000","last_unstale":"2024-09-12T03:03:41.672013+0000","last_undegraded":"2024-09-12T03:03:41.672013+0000","last_fullsized":"2024-09-12T03:03:41.672013+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:35.072323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672214+0000","last_change":"2024-09-12T03:03:07.445674+0000","last_active":"2024-09-12T03:03:41.672214+0000","last_peered":"2024-09-12T03:03:41.672214+0000","last_clean":"2024-09-12T03:03:41.672214+0000","last_became_active":"2024-09-12T03:03:07.445237+0000","last_became_peered":"2024-09-12T03:03:07.445237+0000","last_unstale":"2024-09-12T03:03:41.672214+0000","last_undegraded":"2024-09-12T03:03:41.672214+0000","last_fullsized":"2024-09-12T03:03:41.672214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:56.876238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.810512+0000","last_change":"2024-09-12T03:03:04.629794+0000","last_active":"2024-09-12T03:03:55.810512+0000","last_peered":"2024-09-12T03:03:55.810512+0000","last_clean":"2024-09-12T03:03:55.810512+0000","last_became_active":"2024-09-12T03:03:04.629352+0000","last_became_peered":"2024-09-12T03:03:04.629352+0000","last_unstale":"2024-09-12T03:03:55.810512+0000","last_undegraded":"2024-09-12T03:03:55.810512+0000","last_fullsized":"2024-09-12T03:03:55.810512+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:32.168567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672261+0000","last_change":"2024-09-12T03:03:17.525419+0000","last_active":"2024-09-12T03:03:41.672261+0000","last_peered":"2024-09-12T03:03:41.672261+0000","last_clean":"2024-09-12T03:03:41.672261+0000","last_became_active":"2024-09-12T03:03:17.524894+0000","last_became_peered":"2024-09-12T03:03:17.524894+0000","last_unstale":"2024-09-12T03:03:41.672261+0000","last_undegraded":"2024-09-12T03:03:41.672261+0000","last_fullsized":"2024-09-12T03:03:41.672261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:57.441301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671959+0000","last_change":"2024-09-12T03:03:07.447990+0000","last_active":"2024-09-12T03:03:41.671959+0000","last_peered":"2024-09-12T03:03:41.671959+0000","last_clean":"2024-09-12T03:03:41.671959+0000","last_became_active":"2024-09-12T03:03:07.447648+0000","last_became_peered":"2024-09-12T03:03:07.447648+0000","last_unstale":"2024-09-12T03:03:41.671959+0000","last_undegraded":"2024-09-12T03:03:41.671959+0000","last_fullsized":"2024-09-12T03:03:41.671959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:48.639273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455107+0000","last_change":"2024-09-12T03:03:04.421306+0000","last_active":"2024-09-12T03:03:42.455107+0000","last_peered":"2024-09-12T03:03:42.455107+0000","last_clean":"2024-09-12T03:03:42.455107+0000","last_became_active":"2024-09-12T03:03:04.420688+0000","last_became_peered":"2024-09-12T03:03:04.420688+0000","last_unstale":"2024-09-12T03:03:42.455107+0000","last_undegraded":"2024-09-12T03:03:42.455107+0000","last_fullsized":"2024-09-12T03:03:42.455107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:15.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670487+0000","last_change":"2024-09-12T03:03:17.530457+0000","last_active":"2024-09-12T03:03:41.670487+0000","last_peered":"2024-09-12T03:03:41.670487+0000","last_clean":"2024-09-12T03:03:41.670487+0000","last_became_active":"2024-09-12T03:03:17.530044+0000","last_became_peered":"2024-09-12T03:03:17.530044+0000","last_unstale":"2024-09-12T03:03:41.670487+0000","last_undegraded":"2024-09-12T03:03:41.670487+0000","last_fullsized":"2024-09-12T03:03:41.670487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:01.304089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672473+0000","last_change":"2024-09-12T03:03:07.445935+0000","last_active":"2024-09-12T03:03:41.672473+0000","last_peered":"2024-09-12T03:03:41.672473+0000","last_clean":"2024-09-12T03:03:41.672473+0000","last_became_active":"2024-09-12T03:03:07.444960+0000","last_became_peered":"2024-09-12T03:03:07.444960+0000","last_unstale":"2024-09-12T03:03:41.672473+0000","last_undegraded":"2024-09-12T03:03:41.672473+0000","last_fullsized":"2024-09-12T03:03:41.672473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:53.246923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953867+0000","last_change":"2024-09-12T03:03:04.424147+0000","last_active":"2024-09-12T03:03:41.953867+0000","last_peered":"2024-09-12T03:03:41.953867+0000","last_clean":"2024-09-12T03:03:41.953867+0000","last_became_active":"2024-09-12T03:03:04.423787+0000","last_became_peered":"2024-09-12T03:03:04.423787+0000","last_unstale":"2024-09-12T03:03:41.953867+0000","last_undegraded":"2024-09-12T03:03:41.953867+0000","last_fullsized":"2024-09-12T03:03:41.953867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:48.155556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454133+0000","last_change":"2024-09-12T03:03:17.520974+0000","last_active":"2024-09-12T03:03:42.454133+0000","last_peered":"2024-09-12T03:03:42.454133+0000","last_clean":"2024-09-12T03:03:42.454133+0000","last_became_active":"2024-09-12T03:03:17.520332+0000","last_became_peered":"2024-09-12T03:03:17.520332+0000","last_unstale":"2024-09-12T03:03:42.454133+0000","last_undegraded":"2024-09-12T03:03:42.454133+0000","last_fullsized":"2024-09-12T03:03:42.454133+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:45.061197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456589+0000","last_change":"2024-09-12T03:03:42.456589+0000","last_active":"2024-09-12T03:03:42.456589+0000","last_peered":"2024-09-12T03:03:42.456589+0000","last_clean":"2024-09-12T03:03:42.456589+0000","last_became_active":"2024-09-12T03:03:42.456147+0000","last_became_peered":"2024-09-12T03:03:42.456147+0000","last_unstale":"2024-09-12T03:03:42.456589+0000","last_undegraded":"2024-09-12T03:03:42.456589+0000","last_fullsized":"2024-09-12T03:03:42.456589+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:58.009928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:56.804513+0000","last_change":"2024-09-12T03:03:04.420471+0000","last_active":"2024-09-12T03:03:56.804513+0000","last_peered":"2024-09-12T03:03:56.804513+0000","last_clean":"2024-09-12T03:03:56.804513+0000","last_became_active":"2024-09-12T03:03:04.420093+0000","last_became_peered":"2024-09-12T03:03:04.420093+0000","last_unstale":"2024-09-12T03:03:56.804513+0000","last_undegraded":"2024-09-12T03:03:56.804513+0000","last_fullsized":"2024-09-12T03:03:56.804513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:23.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454371+0000","last_change":"2024-09-12T03:03:17.528073+0000","last_active":"2024-09-12T03:03:42.454371+0000","last_peered":"2024-09-12T03:03:42.454371+0000","last_clean":"2024-09-12T03:03:42.454371+0000","last_became_active":"2024-09-12T03:03:17.527654+0000","last_became_peered":"2024-09-12T03:03:17.527654+0000","last_unstale":"2024-09-12T03:03:42.454371+0000","last_undegraded":"2024-09-12T03:03:42.454371+0000","last_fullsized":"2024-09-12T03:03:42.454371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:45.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454660+0000","last_change":"2024-09-12T03:03:07.455305+0000","last_active":"2024-09-12T03:03:42.454660+0000","last_peered":"2024-09-12T03:03:42.454660+0000","last_clean":"2024-09-12T03:03:42.454660+0000","last_became_active":"2024-09-12T03:03:07.454957+0000","last_became_peered":"2024-09-12T03:03:07.454957+0000","last_unstale":"2024-09-12T03:03:42.454660+0000","last_undegraded":"2024-09-12T03:03:42.454660+0000","last_fullsized":"2024-09-12T03:03:42.454660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:07.152100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":55,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809053+0000","last_change":"2024-09-12T03:03:04.424531+0000","last_active":"2024-09-12T03:03:55.809053+0000","last_peered":"2024-09-12T03:03:55.809053+0000","last_clean":"2024-09-12T03:03:55.809053+0000","last_became_active":"2024-09-12T03:03:04.424230+0000","last_became_peered":"2024-09-12T03:03:04.424230+0000","last_unstale":"2024-09-12T03:03:55.809053+0000","last_undegraded":"2024-09-12T03:03:55.809053+0000","last_fullsized":"2024-09-12T03:03:55.809053+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:01.392262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953859+0000","last_change":"2024-09-12T03:03:17.532131+0000","last_active":"2024-09-12T03:03:41.953859+0000","last_peered":"2024-09-12T03:03:41.953859+0000","last_clean":"2024-09-12T03:03:41.953859+0000","last_became_active":"2024-09-12T03:03:17.531711+0000","last_became_peered":"2024-09-12T03:03:17.531711+0000","last_unstale":"2024-09-12T03:03:41.953859+0000","last_undegraded":"2024-09-12T03:03:41.953859+0000","last_fullsized":"2024-09-12T03:03:41.953859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:15.880954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670194+0000","last_change":"2024-09-12T03:03:07.449101+0000","last_active":"2024-09-12T03:03:41.670194+0000","last_peered":"2024-09-12T03:03:41.670194+0000","last_clean":"2024-09-12T03:03:41.670194+0000","last_became_active":"2024-09-12T03:03:07.448719+0000","last_became_peered":"2024-09-12T03:03:07.448719+0000","last_unstale":"2024-09-12T03:03:41.670194+0000","last_undegraded":"2024-09-12T03:03:41.670194+0000","last_fullsized":"2024-09-12T03:03:41.670194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:45.400939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":66,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:15.842932+0000","last_change":"2024-09-12T03:03:04.422174+0000","last_active":"2024-09-12T03:04:15.842932+0000","last_peered":"2024-09-12T03:04:15.842932+0000","last_clean":"2024-09-12T03:04:15.842932+0000","last_became_active":"2024-09-12T03:03:04.421771+0000","last_became_peered":"2024-09-12T03:03:04.421771+0000","last_unstale":"2024-09-12T03:04:15.842932+0000","last_undegraded":"2024-09-12T03:04:15.842932+0000","last_fullsized":"2024-09-12T03:04:15.842932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:38.149590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175926+0000","last_change":"2024-09-12T03:03:17.535123+0000","last_active":"2024-09-12T03:03:42.175926+0000","last_peered":"2024-09-12T03:03:42.175926+0000","last_clean":"2024-09-12T03:03:42.175926+0000","last_became_active":"2024-09-12T03:03:17.534218+0000","last_became_peered":"2024-09-12T03:03:17.534218+0000","last_unstale":"2024-09-12T03:03:42.175926+0000","last_undegraded":"2024-09-12T03:03:42.175926+0000","last_fullsized":"2024-09-12T03:03:42.175926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:55.870690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671534+0000","last_change":"2024-09-12T03:03:07.453736+0000","last_active":"2024-09-12T03:03:41.671534+0000","last_peered":"2024-09-12T03:03:41.671534+0000","last_clean":"2024-09-12T03:03:41.671534+0000","last_became_active":"2024-09-12T03:03:07.453404+0000","last_became_peered":"2024-09-12T03:03:07.453404+0000","last_unstale":"2024-09-12T03:03:41.671534+0000","last_undegraded":"2024-09-12T03:03:41.671534+0000","last_fullsized":"2024-09-12T03:03:41.671534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:57.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.798006+0000","last_change":"2024-09-12T03:03:04.419663+0000","last_active":"2024-09-12T03:03:54.798006+0000","last_peered":"2024-09-12T03:03:54.798006+0000","last_clean":"2024-09-12T03:03:54.798006+0000","last_became_active":"2024-09-12T03:03:04.419119+0000","last_became_peered":"2024-09-12T03:03:04.419119+0000","last_unstale":"2024-09-12T03:03:54.798006+0000","last_undegraded":"2024-09-12T03:03:54.798006+0000","last_fullsized":"2024-09-12T03:03:54.798006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:11.809683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669223+0000","last_change":"2024-09-12T03:03:17.532686+0000","last_active":"2024-09-12T03:03:41.669223+0000","last_peered":"2024-09-12T03:03:41.669223+0000","last_clean":"2024-09-12T03:03:41.669223+0000","last_became_active":"2024-09-12T03:03:17.532234+0000","last_became_peered":"2024-09-12T03:03:17.532234+0000","last_unstale":"2024-09-12T03:03:41.669223+0000","last_undegraded":"2024-09-12T03:03:41.669223+0000","last_fullsized":"2024-09-12T03:03:41.669223+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:49.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953501+0000","last_change":"2024-09-12T03:03:07.441911+0000","last_active":"2024-09-12T03:03:41.953501+0000","last_peered":"2024-09-12T03:03:41.953501+0000","last_clean":"2024-09-12T03:03:41.953501+0000","last_became_active":"2024-09-12T03:03:07.441323+0000","last_became_peered":"2024-09-12T03:03:07.441323+0000","last_unstale":"2024-09-12T03:03:41.953501+0000","last_undegraded":"2024-09-12T03:03:41.953501+0000","last_fullsized":"2024-09-12T03:03:41.953501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:03.830843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.810725+0000","last_change":"2024-09-12T03:03:04.423648+0000","last_active":"2024-09-12T03:03:55.810725+0000","last_peered":"2024-09-12T03:03:55.810725+0000","last_clean":"2024-09-12T03:03:55.810725+0000","last_became_active":"2024-09-12T03:03:04.423274+0000","last_became_peered":"2024-09-12T03:03:04.423274+0000","last_unstale":"2024-09-12T03:03:55.810725+0000","last_undegraded":"2024-09-12T03:03:55.810725+0000","last_fullsized":"2024-09-12T03:03:55.810725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:50.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454771+0000","last_change":"2024-09-12T03:03:17.522286+0000","last_active":"2024-09-12T03:03:42.454771+0000","last_peered":"2024-09-12T03:03:42.454771+0000","last_clean":"2024-09-12T03:03:42.454771+0000","last_became_active":"2024-09-12T03:03:17.521909+0000","last_became_peered":"2024-09-12T03:03:17.521909+0000","last_unstale":"2024-09-12T03:03:42.454771+0000","last_undegraded":"2024-09-12T03:03:42.454771+0000","last_fullsized":"2024-09-12T03:03:42.454771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:13.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672284+0000","last_change":"2024-09-12T03:03:07.445445+0000","last_active":"2024-09-12T03:03:41.672284+0000","last_peered":"2024-09-12T03:03:41.672284+0000","last_clean":"2024-09-12T03:03:41.672284+0000","last_became_active":"2024-09-12T03:03:07.445086+0000","last_became_peered":"2024-09-12T03:03:07.445086+0000","last_unstale":"2024-09-12T03:03:41.672284+0000","last_undegraded":"2024-09-12T03:03:41.672284+0000","last_fullsized":"2024-09-12T03:03:41.672284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:01.046168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'29","reported_seq":114,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:14.840850+0000","last_change":"2024-09-12T03:03:04.425545+0000","last_active":"2024-09-12T03:04:14.840850+0000","last_peered":"2024-09-12T03:04:14.840850+0000","last_clean":"2024-09-12T03:04:14.840850+0000","last_became_active":"2024-09-12T03:03:04.425141+0000","last_became_peered":"2024-09-12T03:03:04.425141+0000","last_unstale":"2024-09-12T03:04:14.840850+0000","last_undegraded":"2024-09-12T03:04:14.840850+0000","last_fullsized":"2024-09-12T03:04:14.840850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:54.548787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57258,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":78,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175605+0000","last_change":"2024-09-12T03:03:17.538157+0000","last_active":"2024-09-12T03:03:42.175605+0000","last_peered":"2024-09-12T03:03:42.175605+0000","last_clean":"2024-09-12T03:03:42.175605+0000","last_became_active":"2024-09-12T03:03:17.537194+0000","last_became_peered":"2024-09-12T03:03:17.537194+0000","last_unstale":"2024-09-12T03:03:42.175605+0000","last_undegraded":"2024-09-12T03:03:42.175605+0000","last_fullsized":"2024-09-12T03:03:42.175605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:00.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672227+0000","last_change":"2024-09-12T03:03:07.446470+0000","last_active":"2024-09-12T03:03:41.672227+0000","last_peered":"2024-09-12T03:03:41.672227+0000","last_clean":"2024-09-12T03:03:41.672227+0000","last_became_active":"2024-09-12T03:03:07.445975+0000","last_became_peered":"2024-09-12T03:03:07.445975+0000","last_unstale":"2024-09-12T03:03:41.672227+0000","last_undegraded":"2024-09-12T03:03:41.672227+0000","last_fullsized":"2024-09-12T03:03:41.672227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:47.795117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.799739+0000","last_change":"2024-09-12T03:03:04.421833+0000","last_active":"2024-09-12T03:03:54.799739+0000","last_peered":"2024-09-12T03:03:54.799739+0000","last_clean":"2024-09-12T03:03:54.799739+0000","last_became_active":"2024-09-12T03:03:04.421439+0000","last_became_peered":"2024-09-12T03:03:04.421439+0000","last_unstale":"2024-09-12T03:03:54.799739+0000","last_undegraded":"2024-09-12T03:03:54.799739+0000","last_fullsized":"2024-09-12T03:03:54.799739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673561+0000","last_change":"2024-09-12T03:03:17.541327+0000","last_active":"2024-09-12T03:03:41.673561+0000","last_peered":"2024-09-12T03:03:41.673561+0000","last_clean":"2024-09-12T03:03:41.673561+0000","last_became_active":"2024-09-12T03:03:17.540959+0000","last_became_peered":"2024-09-12T03:03:17.540959+0000","last_unstale":"2024-09-12T03:03:41.673561+0000","last_undegraded":"2024-09-12T03:03:41.673561+0000","last_fullsized":"2024-09-12T03:03:41.673561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:16.833511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670609+0000","last_change":"2024-09-12T03:03:07.458806+0000","last_active":"2024-09-12T03:03:41.670609+0000","last_peered":"2024-09-12T03:03:41.670609+0000","last_clean":"2024-09-12T03:03:41.670609+0000","last_became_active":"2024-09-12T03:03:07.458501+0000","last_became_peered":"2024-09-12T03:03:07.458501+0000","last_unstale":"2024-09-12T03:03:41.670609+0000","last_undegraded":"2024-09-12T03:03:41.670609+0000","last_fullsized":"2024-09-12T03:03:41.670609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:54.608552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:54.797761+0000","last_change":"2024-09-12T03:03:41.676350+0000","last_active":"2024-09-12T03:03:54.797761+0000","last_peered":"2024-09-12T03:03:54.797761+0000","last_clean":"2024-09-12T03:03:54.797761+0000","last_became_active":"2024-09-12T03:03:41.675935+0000","last_became_peered":"2024-09-12T03:03:41.675935+0000","last_unstale":"2024-09-12T03:03:54.797761+0000","last_undegraded":"2024-09-12T03:03:54.797761+0000","last_fullsized":"2024-09-12T03:03:54.797761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:11.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175316+0000","last_change":"2024-09-12T03:03:17.533670+0000","last_active":"2024-09-12T03:03:42.175316+0000","last_peered":"2024-09-12T03:03:42.175316+0000","last_clean":"2024-09-12T03:03:42.175316+0000","last_became_active":"2024-09-12T03:03:17.530516+0000","last_became_peered":"2024-09-12T03:03:17.530516+0000","last_unstale":"2024-09-12T03:03:42.175316+0000","last_undegraded":"2024-09-12T03:03:42.175316+0000","last_fullsized":"2024-09-12T03:03:42.175316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:02.228503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672465+0000","last_change":"2024-09-12T03:03:07.448760+0000","last_active":"2024-09-12T03:03:41.672465+0000","last_peered":"2024-09-12T03:03:41.672465+0000","last_clean":"2024-09-12T03:03:41.672465+0000","last_became_active":"2024-09-12T03:03:07.447978+0000","last_became_peered":"2024-09-12T03:03:07.447978+0000","last_unstale":"2024-09-12T03:03:41.672465+0000","last_undegraded":"2024-09-12T03:03:41.672465+0000","last_fullsized":"2024-09-12T03:03:41.672465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.319881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":62,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:13.839379+0000","last_change":"2024-09-12T03:03:04.427752+0000","last_active":"2024-09-12T03:04:13.839379+0000","last_peered":"2024-09-12T03:04:13.839379+0000","last_clean":"2024-09-12T03:04:13.839379+0000","last_became_active":"2024-09-12T03:03:04.427327+0000","last_became_peered":"2024-09-12T03:03:04.427327+0000","last_unstale":"2024-09-12T03:04:13.839379+0000","last_undegraded":"2024-09-12T03:04:13.839379+0000","last_fullsized":"2024-09-12T03:04:13.839379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:51.271870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671398+0000","last_change":"2024-09-12T03:03:17.520463+0000","last_active":"2024-09-12T03:03:41.671398+0000","last_peered":"2024-09-12T03:03:41.671398+0000","last_clean":"2024-09-12T03:03:41.671398+0000","last_became_active":"2024-09-12T03:03:17.520081+0000","last_became_peered":"2024-09-12T03:03:17.520081+0000","last_unstale":"2024-09-12T03:03:41.671398+0000","last_undegraded":"2024-09-12T03:03:41.671398+0000","last_fullsized":"2024-09-12T03:03:41.671398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:56.707115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669877+0000","last_change":"2024-09-12T03:03:17.532373+0000","last_active":"2024-09-12T03:03:41.669877+0000","last_peered":"2024-09-12T03:03:41.669877+0000","last_clean":"2024-09-12T03:03:41.669877+0000","last_became_active":"2024-09-12T03:03:17.525425+0000","last_became_peered":"2024-09-12T03:03:17.525425+0000","last_unstale":"2024-09-12T03:03:41.669877+0000","last_undegraded":"2024-09-12T03:03:41.669877+0000","last_fullsized":"2024-09-12T03:03:41.669877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:29.811678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175798+0000","last_change":"2024-09-12T03:03:07.445936+0000","last_active":"2024-09-12T03:03:42.175798+0000","last_peered":"2024-09-12T03:03:42.175798+0000","last_clean":"2024-09-12T03:03:42.175798+0000","last_became_active":"2024-09-12T03:03:07.445643+0000","last_became_peered":"2024-09-12T03:03:07.445643+0000","last_unstale":"2024-09-12T03:03:42.175798+0000","last_undegraded":"2024-09-12T03:03:42.175798+0000","last_fullsized":"2024-09-12T03:03:42.175798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:07.477841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455384+0000","last_change":"2024-09-12T03:03:04.424947+0000","last_active":"2024-09-12T03:03:42.455384+0000","last_peered":"2024-09-12T03:03:42.455384+0000","last_clean":"2024-09-12T03:03:42.455384+0000","last_became_active":"2024-09-12T03:03:04.424297+0000","last_became_peered":"2024-09-12T03:03:04.424297+0000","last_unstale":"2024-09-12T03:03:42.455384+0000","last_undegraded":"2024-09-12T03:03:42.455384+0000","last_fullsized":"2024-09-12T03:03:42.455384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:11.187945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669389+0000","last_change":"2024-09-12T03:03:17.530616+0000","last_active":"2024-09-12T03:03:41.669389+0000","last_peered":"2024-09-12T03:03:41.669389+0000","last_clean":"2024-09-12T03:03:41.669389+0000","last_became_active":"2024-09-12T03:03:17.530146+0000","last_became_peered":"2024-09-12T03:03:17.530146+0000","last_unstale":"2024-09-12T03:03:41.669389+0000","last_undegraded":"2024-09-12T03:03:41.669389+0000","last_fullsized":"2024-09-12T03:03:41.669389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:26.892311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671245+0000","last_change":"2024-09-12T03:03:07.448084+0000","last_active":"2024-09-12T03:03:41.671245+0000","last_peered":"2024-09-12T03:03:41.671245+0000","last_clean":"2024-09-12T03:03:41.671245+0000","last_became_active":"2024-09-12T03:03:07.447363+0000","last_became_peered":"2024-09-12T03:03:07.447363+0000","last_unstale":"2024-09-12T03:03:41.671245+0000","last_undegraded":"2024-09-12T03:03:41.671245+0000","last_fullsized":"2024-09-12T03:03:41.671245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:07.451892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.809610+0000","last_change":"2024-09-12T03:03:04.430371+0000","last_active":"2024-09-12T03:03:55.809610+0000","last_peered":"2024-09-12T03:03:55.809610+0000","last_clean":"2024-09-12T03:03:55.809610+0000","last_became_active":"2024-09-12T03:03:04.429811+0000","last_became_peered":"2024-09-12T03:03:04.429811+0000","last_unstale":"2024-09-12T03:03:55.809610+0000","last_undegraded":"2024-09-12T03:03:55.809610+0000","last_fullsized":"2024-09-12T03:03:55.809610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:07.443242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:55.807883+0000","last_change":"2024-09-12T03:03:04.422909+0000","last_active":"2024-09-12T03:03:55.807883+0000","last_peered":"2024-09-12T03:03:55.807883+0000","last_clean":"2024-09-12T03:03:55.807883+0000","last_became_active":"2024-09-12T03:03:04.422476+0000","last_became_peered":"2024-09-12T03:03:04.422476+0000","last_unstale":"2024-09-12T03:03:55.807883+0000","last_undegraded":"2024-09-12T03:03:55.807883+0000","last_fullsized":"2024-09-12T03:03:55.807883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.455001+0000","last_change":"2024-09-12T03:03:07.454873+0000","last_active":"2024-09-12T03:03:42.455001+0000","last_peered":"2024-09-12T03:03:42.455001+0000","last_clean":"2024-09-12T03:03:42.455001+0000","last_became_active":"2024-09-12T03:03:07.454521+0000","last_became_peered":"2024-09-12T03:03:07.454521+0000","last_unstale":"2024-09-12T03:03:42.455001+0000","last_undegraded":"2024-09-12T03:03:42.455001+0000","last_fullsized":"2024-09-12T03:03:42.455001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:01.710362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.456398+0000","last_change":"2024-09-12T03:03:42.456398+0000","last_active":"2024-09-12T03:03:42.456398+0000","last_peered":"2024-09-12T03:03:42.456398+0000","last_clean":"2024-09-12T03:03:42.456398+0000","last_became_active":"2024-09-12T03:03:42.455878+0000","last_became_peered":"2024-09-12T03:03:42.455878+0000","last_unstale":"2024-09-12T03:03:42.456398+0000","last_undegraded":"2024-09-12T03:03:42.456398+0000","last_fullsized":"2024-09-12T03:03:42.456398+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:31.760018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:04:14.841222+0000","last_change":"2024-09-12T03:03:04.424039+0000","last_active":"2024-09-12T03:04:14.841222+0000","last_peered":"2024-09-12T03:04:14.841222+0000","last_clean":"2024-09-12T03:04:14.841222+0000","last_became_active":"2024-09-12T03:03:04.422184+0000","last_became_peered":"2024-09-12T03:03:04.422184+0000","last_unstale":"2024-09-12T03:04:14.841222+0000","last_undegraded":"2024-09-12T03:04:14.841222+0000","last_fullsized":"2024-09-12T03:04:14.841222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:07.108262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671925+0000","last_change":"2024-09-12T03:03:07.447753+0000","last_active":"2024-09-12T03:03:41.671925+0000","last_peered":"2024-09-12T03:03:41.671925+0000","last_clean":"2024-09-12T03:03:41.671925+0000","last_became_active":"2024-09-12T03:03:07.447364+0000","last_became_peered":"2024-09-12T03:03:07.447364+0000","last_unstale":"2024-09-12T03:03:41.671925+0000","last_undegraded":"2024-09-12T03:03:41.671925+0000","last_fullsized":"2024-09-12T03:03:41.671925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:11.120666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667317+0000","last_change":"2024-09-12T03:03:17.521454+0000","last_active":"2024-09-12T03:03:41.667317+0000","last_peered":"2024-09-12T03:03:41.667317+0000","last_clean":"2024-09-12T03:03:41.667317+0000","last_became_active":"2024-09-12T03:03:17.521053+0000","last_became_peered":"2024-09-12T03:03:17.521053+0000","last_unstale":"2024-09-12T03:03:41.667317+0000","last_undegraded":"2024-09-12T03:03:41.667317+0000","last_fullsized":"2024-09-12T03:03:41.667317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:57.088465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669038+0000","last_change":"2024-09-12T03:03:07.451140+0000","last_active":"2024-09-12T03:03:41.669038+0000","last_peered":"2024-09-12T03:03:41.669038+0000","last_clean":"2024-09-12T03:03:41.669038+0000","last_became_active":"2024-09-12T03:03:07.450775+0000","last_became_peered":"2024-09-12T03:03:07.450775+0000","last_unstale":"2024-09-12T03:03:41.669038+0000","last_undegraded":"2024-09-12T03:03:41.669038+0000","last_fullsized":"2024-09-12T03:03:41.669038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:12.131080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667313+0000","last_change":"2024-09-12T03:03:07.445602+0000","last_active":"2024-09-12T03:03:41.667313+0000","last_peered":"2024-09-12T03:03:41.667313+0000","last_clean":"2024-09-12T03:03:41.667313+0000","last_became_active":"2024-09-12T03:03:07.445210+0000","last_became_peered":"2024-09-12T03:03:07.445210+0000","last_unstale":"2024-09-12T03:03:41.667313+0000","last_undegraded":"2024-09-12T03:03:41.667313+0000","last_fullsized":"2024-09-12T03:03:41.667313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:48.044322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667017+0000","last_change":"2024-09-12T03:03:07.453006+0000","last_active":"2024-09-12T03:03:41.667017+0000","last_peered":"2024-09-12T03:03:41.667017+0000","last_clean":"2024-09-12T03:03:41.667017+0000","last_became_active":"2024-09-12T03:03:07.452608+0000","last_became_peered":"2024-09-12T03:03:07.452608+0000","last_unstale":"2024-09-12T03:03:41.667017+0000","last_undegraded":"2024-09-12T03:03:41.667017+0000","last_fullsized":"2024-09-12T03:03:41.667017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:22.443868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670588+0000","last_change":"2024-09-12T03:03:07.457284+0000","last_active":"2024-09-12T03:03:41.670588+0000","last_peered":"2024-09-12T03:03:41.670588+0000","last_clean":"2024-09-12T03:03:41.670588+0000","last_became_active":"2024-09-12T03:03:07.456964+0000","last_became_peered":"2024-09-12T03:03:07.456964+0000","last_unstale":"2024-09-12T03:03:41.670588+0000","last_undegraded":"2024-09-12T03:03:41.670588+0000","last_fullsized":"2024-09-12T03:03:41.670588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.525035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953608+0000","last_change":"2024-09-12T03:03:07.451622+0000","last_active":"2024-09-12T03:03:41.953608+0000","last_peered":"2024-09-12T03:03:41.953608+0000","last_clean":"2024-09-12T03:03:41.953608+0000","last_became_active":"2024-09-12T03:03:07.451240+0000","last_became_peered":"2024-09-12T03:03:07.451240+0000","last_unstale":"2024-09-12T03:03:41.953608+0000","last_undegraded":"2024-09-12T03:03:41.953608+0000","last_fullsized":"2024-09-12T03:03:41.953608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:17.357741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669486+0000","last_change":"2024-09-12T03:03:07.450627+0000","last_active":"2024-09-12T03:03:41.669486+0000","last_peered":"2024-09-12T03:03:41.669486+0000","last_clean":"2024-09-12T03:03:41.669486+0000","last_became_active":"2024-09-12T03:03:07.450279+0000","last_became_peered":"2024-09-12T03:03:07.450279+0000","last_unstale":"2024-09-12T03:03:41.669486+0000","last_undegraded":"2024-09-12T03:03:41.669486+0000","last_fullsized":"2024-09-12T03:03:41.669486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:32.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667829+0000","last_change":"2024-09-12T03:03:07.444125+0000","last_active":"2024-09-12T03:03:41.667829+0000","last_peered":"2024-09-12T03:03:41.667829+0000","last_clean":"2024-09-12T03:03:41.667829+0000","last_became_active":"2024-09-12T03:03:07.443799+0000","last_became_peered":"2024-09-12T03:03:07.443799+0000","last_unstale":"2024-09-12T03:03:41.667829+0000","last_undegraded":"2024-09-12T03:03:41.667829+0000","last_fullsized":"2024-09-12T03:03:41.667829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:49.573684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672236+0000","last_change":"2024-09-12T03:03:07.446562+0000","last_active":"2024-09-12T03:03:41.672236+0000","last_peered":"2024-09-12T03:03:41.672236+0000","last_clean":"2024-09-12T03:03:41.672236+0000","last_became_active":"2024-09-12T03:03:07.445539+0000","last_became_peered":"2024-09-12T03:03:07.445539+0000","last_unstale":"2024-09-12T03:03:41.672236+0000","last_undegraded":"2024-09-12T03:03:41.672236+0000","last_fullsized":"2024-09-12T03:03:41.672236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:06.939290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.454554+0000","last_change":"2024-09-12T03:03:07.444899+0000","last_active":"2024-09-12T03:03:42.454554+0000","last_peered":"2024-09-12T03:03:42.454554+0000","last_clean":"2024-09-12T03:03:42.454554+0000","last_became_active":"2024-09-12T03:03:07.443309+0000","last_became_peered":"2024-09-12T03:03:07.443309+0000","last_unstale":"2024-09-12T03:03:42.454554+0000","last_undegraded":"2024-09-12T03:03:42.454554+0000","last_fullsized":"2024-09-12T03:03:42.454554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:06.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671496+0000","last_change":"2024-09-12T03:03:07.455857+0000","last_active":"2024-09-12T03:03:41.671496+0000","last_peered":"2024-09-12T03:03:41.671496+0000","last_clean":"2024-09-12T03:03:41.671496+0000","last_became_active":"2024-09-12T03:03:07.450237+0000","last_became_peered":"2024-09-12T03:03:07.450237+0000","last_unstale":"2024-09-12T03:03:41.671496+0000","last_undegraded":"2024-09-12T03:03:41.671496+0000","last_fullsized":"2024-09-12T03:03:41.671496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:56.338030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672505+0000","last_change":"2024-09-12T03:03:07.445924+0000","last_active":"2024-09-12T03:03:41.672505+0000","last_peered":"2024-09-12T03:03:41.672505+0000","last_clean":"2024-09-12T03:03:41.672505+0000","last_became_active":"2024-09-12T03:03:07.445449+0000","last_became_peered":"2024-09-12T03:03:07.445449+0000","last_unstale":"2024-09-12T03:03:41.672505+0000","last_undegraded":"2024-09-12T03:03:41.672505+0000","last_fullsized":"2024-09-12T03:03:41.672505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:42.040044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953525+0000","last_change":"2024-09-12T03:03:07.449653+0000","last_active":"2024-09-12T03:03:41.953525+0000","last_peered":"2024-09-12T03:03:41.953525+0000","last_clean":"2024-09-12T03:03:41.953525+0000","last_became_active":"2024-09-12T03:03:07.445691+0000","last_became_peered":"2024-09-12T03:03:07.445691+0000","last_unstale":"2024-09-12T03:03:41.953525+0000","last_undegraded":"2024-09-12T03:03:41.953525+0000","last_fullsized":"2024-09-12T03:03:41.953525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:53.210238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669233+0000","last_change":"2024-09-12T03:03:07.453139+0000","last_active":"2024-09-12T03:03:41.669233+0000","last_peered":"2024-09-12T03:03:41.669233+0000","last_clean":"2024-09-12T03:03:41.669233+0000","last_became_active":"2024-09-12T03:03:07.452770+0000","last_became_peered":"2024-09-12T03:03:07.452770+0000","last_unstale":"2024-09-12T03:03:41.669233+0000","last_undegraded":"2024-09-12T03:03:41.669233+0000","last_fullsized":"2024-09-12T03:03:41.669233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:07.247283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.667561+0000","last_change":"2024-09-12T03:03:07.441645+0000","last_active":"2024-09-12T03:03:41.667561+0000","last_peered":"2024-09-12T03:03:41.667561+0000","last_clean":"2024-09-12T03:03:41.667561+0000","last_became_active":"2024-09-12T03:03:07.441191+0000","last_became_peered":"2024-09-12T03:03:07.441191+0000","last_unstale":"2024-09-12T03:03:41.667561+0000","last_undegraded":"2024-09-12T03:03:41.667561+0000","last_fullsized":"2024-09-12T03:03:41.667561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:40.873214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:42.175893+0000","last_change":"2024-09-12T03:03:07.446707+0000","last_active":"2024-09-12T03:03:42.175893+0000","last_peered":"2024-09-12T03:03:42.175893+0000","last_clean":"2024-09-12T03:03:42.175893+0000","last_became_active":"2024-09-12T03:03:07.446301+0000","last_became_peered":"2024-09-12T03:03:07.446301+0000","last_unstale":"2024-09-12T03:03:42.175893+0000","last_undegraded":"2024-09-12T03:03:42.175893+0000","last_fullsized":"2024-09-12T03:03:42.175893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:30.610892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669605+0000","last_change":"2024-09-12T03:03:07.449478+0000","last_active":"2024-09-12T03:03:41.669605+0000","last_peered":"2024-09-12T03:03:41.669605+0000","last_clean":"2024-09-12T03:03:41.669605+0000","last_became_active":"2024-09-12T03:03:07.448830+0000","last_became_peered":"2024-09-12T03:03:07.448830+0000","last_unstale":"2024-09-12T03:03:41.669605+0000","last_undegraded":"2024-09-12T03:03:41.669605+0000","last_fullsized":"2024-09-12T03:03:41.669605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:55.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.670382+0000","last_change":"2024-09-12T03:03:07.449866+0000","last_active":"2024-09-12T03:03:41.670382+0000","last_peered":"2024-09-12T03:03:41.670382+0000","last_clean":"2024-09-12T03:03:41.670382+0000","last_became_active":"2024-09-12T03:03:07.448934+0000","last_became_peered":"2024-09-12T03:03:07.448934+0000","last_unstale":"2024-09-12T03:03:41.670382+0000","last_undegraded":"2024-09-12T03:03:41.670382+0000","last_fullsized":"2024-09-12T03:03:41.670382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:24.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.673404+0000","last_change":"2024-09-12T03:03:41.673404+0000","last_active":"2024-09-12T03:03:41.673404+0000","last_peered":"2024-09-12T03:03:41.673404+0000","last_clean":"2024-09-12T03:03:41.673404+0000","last_became_active":"2024-09-12T03:03:41.673022+0000","last_became_peered":"2024-09-12T03:03:41.673022+0000","last_unstale":"2024-09-12T03:03:41.673404+0000","last_undegraded":"2024-09-12T03:03:41.673404+0000","last_fullsized":"2024-09-12T03:03:41.673404+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:44.083090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669757+0000","last_change":"2024-09-12T03:03:07.455058+0000","last_active":"2024-09-12T03:03:41.669757+0000","last_peered":"2024-09-12T03:03:41.669757+0000","last_clean":"2024-09-12T03:03:41.669757+0000","last_became_active":"2024-09-12T03:03:07.449331+0000","last_became_peered":"2024-09-12T03:03:07.449331+0000","last_unstale":"2024-09-12T03:03:41.669757+0000","last_undegraded":"2024-09-12T03:03:41.669757+0000","last_fullsized":"2024-09-12T03:03:41.669757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:30.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.953519+0000","last_change":"2024-09-12T03:03:07.447193+0000","last_active":"2024-09-12T03:03:41.953519+0000","last_peered":"2024-09-12T03:03:41.953519+0000","last_clean":"2024-09-12T03:03:41.953519+0000","last_became_active":"2024-09-12T03:03:07.445206+0000","last_became_peered":"2024-09-12T03:03:07.445206+0000","last_unstale":"2024-09-12T03:03:41.953519+0000","last_undegraded":"2024-09-12T03:03:41.953519+0000","last_fullsized":"2024-09-12T03:03:41.953519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:28.919293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.674184+0000","last_change":"2024-09-12T03:03:07.442621+0000","last_active":"2024-09-12T03:03:41.674184+0000","last_peered":"2024-09-12T03:03:41.674184+0000","last_clean":"2024-09-12T03:03:41.674184+0000","last_became_active":"2024-09-12T03:03:07.441124+0000","last_became_peered":"2024-09-12T03:03:07.441124+0000","last_unstale":"2024-09-12T03:03:41.674184+0000","last_undegraded":"2024-09-12T03:03:41.674184+0000","last_fullsized":"2024-09-12T03:03:41.674184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:08.442627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672123+0000","last_change":"2024-09-12T03:03:07.443619+0000","last_active":"2024-09-12T03:03:41.672123+0000","last_peered":"2024-09-12T03:03:41.672123+0000","last_clean":"2024-09-12T03:03:41.672123+0000","last_became_active":"2024-09-12T03:03:07.443290+0000","last_became_peered":"2024-09-12T03:03:07.443290+0000","last_unstale":"2024-09-12T03:03:41.672123+0000","last_undegraded":"2024-09-12T03:03:41.672123+0000","last_fullsized":"2024-09-12T03:03:41.672123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:43.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.877152+0000","last_change":"2024-09-12T03:03:07.454164+0000","last_active":"2024-09-12T03:03:41.877152+0000","last_peered":"2024-09-12T03:03:41.877152+0000","last_clean":"2024-09-12T03:03:41.877152+0000","last_became_active":"2024-09-12T03:03:07.453835+0000","last_became_peered":"2024-09-12T03:03:07.453835+0000","last_unstale":"2024-09-12T03:03:41.877152+0000","last_undegraded":"2024-09-12T03:03:41.877152+0000","last_fullsized":"2024-09-12T03:03:41.877152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:00.494720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.672224+0000","last_change":"2024-09-12T03:03:07.446854+0000","last_active":"2024-09-12T03:03:41.672224+0000","last_peered":"2024-09-12T03:03:41.672224+0000","last_clean":"2024-09-12T03:03:41.672224+0000","last_became_active":"2024-09-12T03:03:07.446411+0000","last_became_peered":"2024-09-12T03:03:07.446411+0000","last_unstale":"2024-09-12T03:03:41.672224+0000","last_undegraded":"2024-09-12T03:03:41.672224+0000","last_fullsized":"2024-09-12T03:03:41.672224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:54.917686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671209+0000","last_change":"2024-09-12T03:03:07.448332+0000","last_active":"2024-09-12T03:03:41.671209+0000","last_peered":"2024-09-12T03:03:41.671209+0000","last_clean":"2024-09-12T03:03:41.671209+0000","last_became_active":"2024-09-12T03:03:07.447776+0000","last_became_peered":"2024-09-12T03:03:07.447776+0000","last_unstale":"2024-09-12T03:03:41.671209+0000","last_undegraded":"2024-09-12T03:03:41.671209+0000","last_fullsized":"2024-09-12T03:03:41.671209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:08.531088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671147+0000","last_change":"2024-09-12T03:03:07.449781+0000","last_active":"2024-09-12T03:03:41.671147+0000","last_peered":"2024-09-12T03:03:41.671147+0000","last_clean":"2024-09-12T03:03:41.671147+0000","last_became_active":"2024-09-12T03:03:07.449359+0000","last_became_peered":"2024-09-12T03:03:07.449359+0000","last_unstale":"2024-09-12T03:03:41.671147+0000","last_undegraded":"2024-09-12T03:03:41.671147+0000","last_fullsized":"2024-09-12T03:03:41.671147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:11.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.877017+0000","last_change":"2024-09-12T03:03:07.453881+0000","last_active":"2024-09-12T03:03:41.877017+0000","last_peered":"2024-09-12T03:03:41.877017+0000","last_clean":"2024-09-12T03:03:41.877017+0000","last_became_active":"2024-09-12T03:03:07.453455+0000","last_became_peered":"2024-09-12T03:03:07.453455+0000","last_unstale":"2024-09-12T03:03:41.877017+0000","last_undegraded":"2024-09-12T03:03:41.877017+0000","last_fullsized":"2024-09-12T03:03:41.877017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:18:46.797598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.671873+0000","last_change":"2024-09-12T03:03:07.459332+0000","last_active":"2024-09-12T03:03:41.671873+0000","last_peered":"2024-09-12T03:03:41.671873+0000","last_clean":"2024-09-12T03:03:41.671873+0000","last_became_active":"2024-09-12T03:03:07.459032+0000","last_became_peered":"2024-09-12T03:03:07.459032+0000","last_unstale":"2024-09-12T03:03:41.671873+0000","last_undegraded":"2024-09-12T03:03:41.671873+0000","last_fullsized":"2024-09-12T03:03:41.671873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:16.702782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.674083+0000","last_change":"2024-09-12T03:03:07.441696+0000","last_active":"2024-09-12T03:03:41.674083+0000","last_peered":"2024-09-12T03:03:41.674083+0000","last_clean":"2024-09-12T03:03:41.674083+0000","last_became_active":"2024-09-12T03:03:07.440575+0000","last_became_peered":"2024-09-12T03:03:07.440575+0000","last_unstale":"2024-09-12T03:03:41.674083+0000","last_undegraded":"2024-09-12T03:03:41.674083+0000","last_fullsized":"2024-09-12T03:03:41.674083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:26.049893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669856+0000","last_change":"2024-09-12T03:03:07.454945+0000","last_active":"2024-09-12T03:03:41.669856+0000","last_peered":"2024-09-12T03:03:41.669856+0000","last_clean":"2024-09-12T03:03:41.669856+0000","last_became_active":"2024-09-12T03:03:07.452368+0000","last_became_peered":"2024-09-12T03:03:07.452368+0000","last_unstale":"2024-09-12T03:03:41.669856+0000","last_undegraded":"2024-09-12T03:03:41.669856+0000","last_fullsized":"2024-09-12T03:03:41.669856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:21.169859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669980+0000","last_change":"2024-09-12T03:03:07.454230+0000","last_active":"2024-09-12T03:03:41.669980+0000","last_peered":"2024-09-12T03:03:41.669980+0000","last_clean":"2024-09-12T03:03:41.669980+0000","last_became_active":"2024-09-12T03:03:07.453913+0000","last_became_peered":"2024-09-12T03:03:07.453913+0000","last_unstale":"2024-09-12T03:03:41.669980+0000","last_undegraded":"2024-09-12T03:03:41.669980+0000","last_fullsized":"2024-09-12T03:03:41.669980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:56.264048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-12T03:03:41.669558+0000","last_change":"2024-09-12T03:03:07.449227+0000","last_active":"2024-09-12T03:03:41.669558+0000","last_peered":"2024-09-12T03:03:41.669558+0000","last_clean":"2024-09-12T03:03:41.669558+0000","last_became_active":"2024-09-12T03:03:07.448605+0000","last_became_peered":"2024-09-12T03:03:07.448605+0000","last_unstale":"2024-09-12T03:03:41.669558+0000","last_undegraded":"2024-09-12T03:03:41.669558+0000","last_fullsized":"2024-09-12T03:03:41.669558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:06.420752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61261,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":144,"num_read_kb":136,"num_write":160,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":183783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":535416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710751,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32900,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712252,"statfs":{"total":95995035648,"available":95961346048,"internally_reserved":0,"allocated":1757184,"data_stored":1179942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874276,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":539748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135915,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":535572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1683456,"data_stored":1131483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528373,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":530828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1127071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":535478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215173,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":531022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1155072,"data_stored":589739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":587996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":532216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:04:19.313 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:04:19.314 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:04:19.314 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:04:19.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:19 smithi057 ceph-mon[30361]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T03:04:19.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:19 smithi057 ceph-mon[30361]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:19 smithi204 ceph-mon[35013]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T03:04:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:19 smithi204 ceph-mon[35013]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:19.697 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:19 smithi181 ceph-mon[35320]: pgmap v343: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T03:04:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:19 smithi181 ceph-mon[35320]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:04:20.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 -- 172.21.15.57:0/359957827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 msgr2=0x7f39bc1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 --2- 172.21.15.57:0/359957827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc1053e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009900 tx=0x7f39a402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 -- 172.21.15.57:0/359957827 shutdown_connections 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 --2- 172.21.15.57:0/359957827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc105920 0x7f39bc10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 --2- 172.21.15.57:0/359957827 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc1053e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 --2- 172.21.15.57:0/359957827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc103e00 0x7f39bc104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.296+0000 7f39c0c00640 1 -- 172.21.15.57:0/359957827 >> 172.21.15.57:0/359957827 conn(0x7f39bc0ff5b0 msgr2=0x7f39bc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.297+0000 7f39c0c00640 1 -- 172.21.15.57:0/359957827 shutdown_connections 2024-09-12T03:04:20.298 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.297+0000 7f39c0c00640 1 -- 172.21.15.57:0/359957827 wait complete. 2024-09-12T03:04:20.299 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.297+0000 7f39c0c00640 1 Processor -- start 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 -- start start 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc103e00 0x7f39bc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 0x7f39bc1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10fab0 con 0x7f39bc104f60 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10f930 con 0x7f39bc105920 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39c0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc10fc30 con 0x7f39bc103e00 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47952/0 (socket says 172.21.15.57:47952) 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39b3e00640 1 -- 172.21.15.57:0/2868497819 learned_addr learned my addr 172.21.15.57:0/2868497819 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:20.300 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39ba800640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 0x7f39bc1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.298+0000 7f39b9e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc103e00 0x7f39bc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 -- 172.21.15.57:0/2868497819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc103e00 msgr2=0x7f39bc19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc103e00 0x7f39bc19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 -- 172.21.15.57:0/2868497819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 msgr2=0x7f39bc1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 0x7f39bc1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 -- 172.21.15.57:0/2868497819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc1a1d30 con 0x7f39bc104f60 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39ba800640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 0x7f39bc1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:20.301 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b3e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39a4031610 tx=0x7f39a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.299+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a403f030 con 0x7f39bc104f60 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.300+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a4003ad0 con 0x7f39bc104f60 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.300+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4035070 con 0x7f39bc104f60 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.300+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc1a1fc0 con 0x7f39bc104f60 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.300+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc1a2360 con 0x7f39bc104f60 2024-09-12T03:04:20.302 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.301+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc10b1c0 con 0x7f39bc104f60 2024-09-12T03:04:20.303 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.303+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39a4042030 con 0x7f39bc104f60 2024-09-12T03:04:20.304 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.304+0000 7f39b2a00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 0x7f39a00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:20.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.304+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f39a40efaa0 con 0x7f39bc104f60 2024-09-12T03:04:20.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.304+0000 7f39b9e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 0x7f39a00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:20.305 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.305+0000 7f39b9e00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 0x7f39a00a5180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009bd0 tx=0x7f39ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:20.308 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.308+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39a40bbe90 con 0x7f39bc104f60 2024-09-12T03:04:20.559 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.559+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f39bc02d7c0 con 0x7f39bc104f60 2024-09-12T03:04:20.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.565+0000 7f39b2a00640 1 -- 172.21.15.57:0/2868497819 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f39a40c0190 con 0x7f39bc104f60 2024-09-12T03:04:20.566 INFO:teuthology.orchestra.run.smithi057.stdout:[client.0] 2024-09-12T03:04:20.566 INFO:teuthology.orchestra.run.smithi057.stdout: key = AQA0WuJmbtdpIRAACRzKxXlJbyoFxq6jvNoC1g== 2024-09-12T03:04:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 msgr2=0x7f39a00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:20.568 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 0x7f39a00a5180 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009bd0 tx=0x7f39ac005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 msgr2=0x7f39bc19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39a4031610 tx=0x7f39a4004270 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 shutdown_connections 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f39a00a2cc0 0x7f39a00a5180 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39bc105920 0x7f39bc1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f39bc104f60 0x7f39bc19a640 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 --2- 172.21.15.57:0/2868497819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39bc103e00 0x7f39bc19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 >> 172.21.15.57:0/2868497819 conn(0x7f39bc0ff5b0 msgr2=0x7f39bc108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 shutdown_connections 2024-09-12T03:04:20.569 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:20.568+0000 7f39c0c00640 1 -- 172.21.15.57:0/2868497819 wait complete. 2024-09-12T03:04:21.302 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:04:21.302 DEBUG:teuthology.orchestra.run.smithi057:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:04:21.302 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:04:21.384 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:04:21.384 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:04:21.384 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:04:21.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:21 smithi057 ceph-mon[30361]: pgmap v344: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:04:21.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:21 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:04:21.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:21 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:04:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:21 smithi204 ceph-mon[35013]: pgmap v344: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:04:21.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:21 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:04:21.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:21 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:04:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:21 smithi181 ceph-mon[35320]: pgmap v344: 129 pgs: 129 active+clean; 637 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-12T03:04:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:21 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:04:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:21 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2868497819' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:04:21.803 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:22.424 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.423+0000 7fe9e7200640 1 -- 172.21.15.57:0/2017806897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0076720 msgr2=0x7fe9e0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.423+0000 7fe9e7200640 1 --2- 172.21.15.57:0/2017806897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0076720 0x7fe9e0076b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009900 tx=0x7fe9d002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 -- 172.21.15.57:0/2017806897 shutdown_connections 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 --2- 172.21.15.57:0/2017806897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0112a50 0x7fe9e0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 --2- 172.21.15.57:0/2017806897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 0x7fe9e0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 --2- 172.21.15.57:0/2017806897 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0076720 0x7fe9e0076b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 -- 172.21.15.57:0/2017806897 >> 172.21.15.57:0/2017806897 conn(0x7fe9e00ffa50 msgr2=0x7fe9e0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 -- 172.21.15.57:0/2017806897 shutdown_connections 2024-09-12T03:04:22.425 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.424+0000 7fe9e7200640 1 -- 172.21.15.57:0/2017806897 wait complete. 2024-09-12T03:04:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.425+0000 7fe9e7200640 1 Processor -- start 2024-09-12T03:04:22.426 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.425+0000 7fe9e7200640 1 -- start start 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0076720 0x7fe9e01a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 0x7fe9e01a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0118610 con 0x7fe9e0112a50 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0118490 con 0x7fe9e0076720 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0118790 con 0x7fe9e0077040 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.427 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47976/0 (socket says 172.21.15.57:47976) 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 -- 172.21.15.57:0/527868602 learned_addr learned my addr 172.21.15.57:0/527868602 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9d7400640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 0x7fe9e01a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9dfe00640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0076720 0x7fe9e01a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 -- 172.21.15.57:0/527868602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 msgr2=0x7fe9e01a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 0x7fe9e01a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 -- 172.21.15.57:0/527868602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0076720 msgr2=0x7fe9e01a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0076720 0x7fe9e01a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.426+0000 7fe9e5200640 1 -- 172.21.15.57:0/527868602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e019f0f0 con 0x7fe9e0112a50 2024-09-12T03:04:22.428 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9e5200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800eb10 tx=0x7fe9d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8004b20 con 0x7fe9e0112a50 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e019f3e0 con 0x7fe9e0112a50 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d8004cc0 con 0x7fe9e0112a50 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8010790 con 0x7fe9e0112a50 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.427+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e01ab6d0 con 0x7fe9e0112a50 2024-09-12T03:04:22.429 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.428+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e0076b00 con 0x7fe9e0112a50 2024-09-12T03:04:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.430+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe9d8010930 con 0x7fe9e0112a50 2024-09-12T03:04:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.432+0000 7fe9dd600640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 0x7fe9c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.432+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe9d80cfcf0 con 0x7fe9e0112a50 2024-09-12T03:04:22.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.432+0000 7fe9dfe00640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 0x7fe9c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.433 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.433+0000 7fe9dfe00640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 0x7fe9c40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.434 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.434+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9d809c730 con 0x7fe9e0112a50 2024-09-12T03:04:22.673 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.672+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe9e019fef0 con 0x7fe9e0112a50 2024-09-12T03:04:22.678 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.678+0000 7fe9dd600640 1 -- 172.21.15.57:0/527868602 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7fe9e019fef0 con 0x7fe9e0112a50 2024-09-12T03:04:22.682 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.681+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 msgr2=0x7fe9c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.681+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 0x7fe9c40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006010 tx=0x7fe9d0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 msgr2=0x7fe9e019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800eb10 tx=0x7fe9d800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 shutdown_connections 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9c40a2c10 0x7fe9c40a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe9e0112a50 0x7fe9e019e9c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.684 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e0077040 0x7fe9e01a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 --2- 172.21.15.57:0/527868602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9e0076720 0x7fe9e01a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 >> 172.21.15.57:0/527868602 conn(0x7fe9e00ffa50 msgr2=0x7fe9e0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 shutdown_connections 2024-09-12T03:04:22.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:22.682+0000 7fe9e7200640 1 -- 172.21.15.57:0/527868602 wait complete. 2024-09-12T03:04:23.575 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6800","nonce":480086157},{"type":"v1","addr":"172.21.15.57:6801","nonce":480086157}]},"active_addr":"172.21.15.57:6801/480086157","active_change":"2024-09-12T02:55:39.619615+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.57:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":1354301616}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2696288462}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":2870728447}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":1679473944}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.57:0","nonce":3214677462}]}]} 2024-09-12T03:04:23.581 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:04:23.582 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:04:23.582 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:04:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:23 smithi181 ceph-mon[35320]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:04:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:23 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/527868602' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:04:23.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:23 smithi057 ceph-mon[30361]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:04:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:23 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/527868602' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:04:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:23 smithi204 ceph-mon[35013]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-09-12T03:04:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:23 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/527868602' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:04:23.962 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 -- 172.21.15.57:0/2414271380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 msgr2=0x7f361c114c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 --2- 172.21.15.57:0/2414271380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c114c50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3614009860 tx=0x7f361402a340 comp rx=0 tx=0).stop 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 -- 172.21.15.57:0/2414271380 shutdown_connections 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 --2- 172.21.15.57:0/2414271380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c114c50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 --2- 172.21.15.57:0/2414271380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 --2- 172.21.15.57:0/2414271380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 -- 172.21.15.57:0/2414271380 >> 172.21.15.57:0/2414271380 conn(0x7f361c0ff880 msgr2=0x7f361c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 -- 172.21.15.57:0/2414271380 shutdown_connections 2024-09-12T03:04:24.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.542+0000 7f361be00640 1 -- 172.21.15.57:0/2414271380 wait complete. 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 Processor -- start 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 -- start start 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1183b0 con 0x7f361c077040 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f361c118230 con 0x7f361c112860 2024-09-12T03:04:24.545 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.543+0000 7f361be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f361c118530 con 0x7f361c076720 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f3612000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f3612000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c1a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48000/0 (socket says 172.21.15.57:48000) 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:50374/0 (socket says 172.21.15.57:50374) 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f3612000640 1 -- 172.21.15.57:0/621185392 learned_addr learned my addr 172.21.15.57:0/621185392 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361b400640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 -- 172.21.15.57:0/621185392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 msgr2=0x7f361c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 -- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 msgr2=0x7f361c1a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c1a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.544+0000 7f361aa00640 1 -- 172.21.15.57:0/621185392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c19ef30 con 0x7f361c076720 2024-09-12T03:04:24.546 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f361aa00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f360c009fd0 tx=0x7f360c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c004b20 con 0x7f361c076720 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c19f220 con 0x7f361c076720 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f360c004cc0 con 0x7f361c076720 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c0107b0 con 0x7f361c076720 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.545+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1ab570 con 0x7f361c076720 2024-09-12T03:04:24.547 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.546+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c077e60 con 0x7f361c076720 2024-09-12T03:04:24.548 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.548+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f360c005020 con 0x7f361c076720 2024-09-12T03:04:24.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.549+0000 7f3613e00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 0x7f36000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.549+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f360c0cfc70 con 0x7f361c076720 2024-09-12T03:04:24.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.550+0000 7f3612000640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 0x7f36000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.550 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.550+0000 7f3612000640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 0x7f36000a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36040096c0 tx=0x7f3604009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.552+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f360c09c6b0 con 0x7f361c076720 2024-09-12T03:04:24.744 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.743+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f361c1ab740 con 0x7f361c076720 2024-09-12T03:04:24.745 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.745+0000 7f3613e00640 1 -- 172.21.15.57:0/621185392 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7f360c09bfb0 con 0x7f361c076720 2024-09-12T03:04:24.746 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:24.746 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:41.651986+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:04:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 msgr2=0x7f36000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 0x7f36000a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36040096c0 tx=0x7f3604009260 comp rx=0 tx=0).stop 2024-09-12T03:04:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 msgr2=0x7f361c1a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f360c009fd0 tx=0x7f360c00ef90 comp rx=0 tx=0).stop 2024-09-12T03:04:24.748 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 shutdown_connections 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f36000a2c90 0x7f36000a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f361c112860 0x7f361c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f361c077040 0x7f361c1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 --2- 172.21.15.57:0/621185392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f361c076720 0x7f361c1a45f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.747+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 >> 172.21.15.57:0/621185392 conn(0x7f361c0ff880 msgr2=0x7f361c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.748+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 shutdown_connections 2024-09-12T03:04:24.749 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:24.748+0000 7f361be00640 1 -- 172.21.15.57:0/621185392 wait complete. 2024-09-12T03:04:25.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:25 smithi057 ceph-mon[30361]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-12T03:04:25.684 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:25 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/621185392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:25.686 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:04:25.687 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:04:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:25 smithi181 ceph-mon[35320]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-12T03:04:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:25 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/621185392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:25 smithi204 ceph-mon[35013]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-12T03:04:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:25 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/621185392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:26.071 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:26.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 -- 172.21.15.57:0/2569355347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb60108f60 msgr2=0x7fbb60110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:26.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 --2- 172.21.15.57:0/2569355347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb60108f60 0x7fbb60110ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009ca0 tx=0x7fbb5802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 -- 172.21.15.57:0/2569355347 shutdown_connections 2024-09-12T03:04:26.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 --2- 172.21.15.57:0/2569355347 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb60108f60 0x7fbb60110ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 --2- 172.21.15.57:0/2569355347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb601047e0 0x7fbb60108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 --2- 172.21.15.57:0/2569355347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60103e30 0x7fbb60104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 -- 172.21.15.57:0/2569355347 >> 172.21.15.57:0/2569355347 conn(0x7fbb600ff8d0 msgr2=0x7fbb60101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:26.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.652+0000 7fbb65e00640 1 -- 172.21.15.57:0/2569355347 shutdown_connections 2024-09-12T03:04:26.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.653+0000 7fbb65e00640 1 -- 172.21.15.57:0/2569355347 wait complete. 2024-09-12T03:04:26.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.653+0000 7fbb65e00640 1 Processor -- start 2024-09-12T03:04:26.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 -- start start 2024-09-12T03:04:26.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 0x7fbb6019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 0x7fbb6019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60114010 con 0x7fbb601047e0 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60113e90 con 0x7fbb60108f60 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60114190 con 0x7fbb60103e30 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 0x7fbb6019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48020/0 (socket says 172.21.15.57:48020) 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb5fe00640 1 -- 172.21.15.57:0/3155483638 learned_addr learned my addr 172.21.15.57:0/3155483638 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.654+0000 7fbb65400640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 0x7fbb6019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 -- 172.21.15.57:0/3155483638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 msgr2=0x7fbb6019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 0x7fbb6019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 -- 172.21.15.57:0/3155483638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 msgr2=0x7fbb6019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 0x7fbb6019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 -- 172.21.15.57:0/3155483638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6019f3b0 con 0x7fbb601047e0 2024-09-12T03:04:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb65400640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 0x7fbb6019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb64a00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 0x7fbb6019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:26.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5fe00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00e9f0 tx=0x7fbb4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00cd80 con 0x7fbb601047e0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c005500 con 0x7fbb601047e0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c010690 con 0x7fbb601047e0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.655+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6019f6a0 con 0x7fbb601047e0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.656+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601a6f80 con 0x7fbb601047e0 2024-09-12T03:04:26.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.657+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb60104210 con 0x7fbb601047e0 2024-09-12T03:04:26.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.659+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbb4c0046c0 con 0x7fbb601047e0 2024-09-12T03:04:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.661+0000 7fbb5d600640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 0x7fbb480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.661+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fbb4c0cf9e0 con 0x7fbb601047e0 2024-09-12T03:04:26.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.661+0000 7fbb64a00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 0x7fbb480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:26.662 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.662+0000 7fbb64a00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 0x7fbb480a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:26.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.665+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb4c09c420 con 0x7fbb601047e0 2024-09-12T03:04:26.853 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.852+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb601a7260 con 0x7fbb601047e0 2024-09-12T03:04:26.854 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.854+0000 7fbb5d600640 1 -- 172.21.15.57:0/3155483638 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24723 (secure 0 0 0) 0x7fbb4c09bd20 con 0x7fbb601047e0 2024-09-12T03:04:26.855 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:04:26.855 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":94,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:03:41.651986+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:04:26.857 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 msgr2=0x7fbb480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 0x7fbb480a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb54005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 msgr2=0x7fbb6019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00e9f0 tx=0x7fbb4c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 shutdown_connections 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbb480a2c10 0x7fbb480a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb60108f60 0x7fbb6019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbb601047e0 0x7fbb6019a940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 --2- 172.21.15.57:0/3155483638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb60103e30 0x7fbb6019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:26.858 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 >> 172.21.15.57:0/3155483638 conn(0x7fbb600ff8d0 msgr2=0x7fbb60101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:26.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 shutdown_connections 2024-09-12T03:04:26.859 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:26.857+0000 7fbb65e00640 1 -- 172.21.15.57:0/3155483638 wait complete. 2024-09-12T03:04:27.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:27 smithi057 ceph-mon[30361]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T03:04:27.576 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:27 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3155483638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:27.616 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:04:27.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:04:27.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:04:27.617 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:04:27.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:04:27.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:04:27.618 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:04:27.619 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:04:27.619 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:04:27.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:04:27.620 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:04:27.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:04:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:27 smithi181 ceph-mon[35320]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T03:04:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:27 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3155483638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:27 smithi204 ceph-mon[35013]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-12T03:04:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:27 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3155483638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:28.649 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.667 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.671 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.686 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.696 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.773 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.852 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.855 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:29.384 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:29.693 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:29 smithi057 ceph-mon[30361]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:04:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:29 smithi181 ceph-mon[35320]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:04:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:29 smithi204 ceph-mon[35013]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-12T03:04:29.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 -- 172.21.15.57:0/2142929414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 msgr2=0x7fd820100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 --2- 172.21.15.57:0/2142929414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820100ee0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd808009860 tx=0x7fd80802a340 comp rx=0 tx=0).stop 2024-09-12T03:04:29.909 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 -- 172.21.15.57:0/2142929414 shutdown_connections 2024-09-12T03:04:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 --2- 172.21.15.57:0/2142929414 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd820101420 0x7fd82010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 --2- 172.21.15.57:0/2142929414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820100ee0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 --2- 172.21.15.57:0/2142929414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820106a80 0x7fd820106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 -- 172.21.15.57:0/2142929414 >> 172.21.15.57:0/2142929414 conn(0x7fd8200fc7a0 msgr2=0x7fd8200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.910 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.908+0000 7fd824a00640 1 -- 172.21.15.57:0/2142929414 shutdown_connections 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.909+0000 7fd824a00640 1 -- 172.21.15.57:0/2142929414 wait complete. 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 Processor -- start 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 -- start start 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820101420 0x7fd8201113d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd820106a80 0x7fd820111af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.911 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd82010fd70 con 0x7fd820106a80 2024-09-12T03:04:29.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd82010fbf0 con 0x7fd820100a80 2024-09-12T03:04:29.912 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.910+0000 7fd824a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd82010fef0 con 0x7fd820101420 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd820106a80 0x7fd820111af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd815400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820101420 0x7fd8201113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820115280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58726/0 (socket says 172.21.15.57:58726) 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81de00640 1 -- 172.21.15.57:0/85615420 learned_addr learned my addr 172.21.15.57:0/85615420 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81de00640 1 -- 172.21.15.57:0/85615420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820101420 msgr2=0x7fd8201113d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.911+0000 7fd81de00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd820101420 0x7fd8201113d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.913 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd81de00640 1 -- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd820106a80 msgr2=0x7fd820111af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd81de00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd820106a80 0x7fd820111af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd81de00640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8201aad50 con 0x7fd820100a80 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd81de00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820115280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd81000e9f0 tx=0x7fd81000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81000cd80 con 0x7fd820100a80 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd810005500 con 0x7fd820100a80 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8201aafe0 con 0x7fd820100a80 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.912+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd810010690 con 0x7fd820100a80 2024-09-12T03:04:29.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.913+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8201ab3e0 con 0x7fd820100a80 2024-09-12T03:04:29.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.913+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd820104f70 con 0x7fd820100a80 2024-09-12T03:04:29.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.916+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd8100046c0 con 0x7fd820100a80 2024-09-12T03:04:29.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd817400640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd8040a2c10 0x7fd8040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fd8100cfd00 con 0x7fd820100a80 2024-09-12T03:04:29.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd815400640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd8040a2c10 0x7fd8040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd817400640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] conn(0x7fd8040adde0 0x7fd8040b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd81000be50 con 0x7fd8040adde0 2024-09-12T03:04:29.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.917+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd8100d00b0 con 0x7fd820100a80 2024-09-12T03:04:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.918+0000 7fd81e800640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] conn(0x7fd8040adde0 0x7fd8040b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.918+0000 7fd815400640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd8040a2c10 0x7fd8040a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd80802a930 tx=0x7fd8080056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.918+0000 7fd81e800640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] conn(0x7fd8040adde0 0x7fd8040b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.919+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== osd.4 v2:172.21.15.181:6808/3730474158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd81000be50 con 0x7fd8040adde0 2024-09-12T03:04:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.934+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 --> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8201021c0 con 0x7fd8040adde0 2024-09-12T03:04:29.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd817400640 1 -- 172.21.15.57:0/85615420 <== osd.4 v2:172.21.15.181:6808/3730474158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd8201021c0 con 0x7fd8040adde0 2024-09-12T03:04:29.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] conn(0x7fd8040adde0 msgr2=0x7fd8040b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] conn(0x7fd8040adde0 0x7fd8040b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd8040a2c10 msgr2=0x7fd8040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd8040a2c10 0x7fd8040a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd80802a930 tx=0x7fd8080056d0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 msgr2=0x7fd820115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd824a00640 1 --2- 172.21.15.57:0/85615420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd820100a80 0x7fd820115280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd81000e9f0 tx=0x7fd81000edf0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.935+0000 7fd81e800640 1 -- 172.21.15.57:0/85615420 reap_dead start 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.936+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 shutdown_connections 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.936+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 >> 172.21.15.57:0/85615420 conn(0x7fd8200fc7a0 msgr2=0x7fd820104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.936+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 shutdown_connections 2024-09-12T03:04:29.937 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.936+0000 7fd824a00640 1 -- 172.21.15.57:0/85615420 wait complete. 2024-09-12T03:04:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.807+0000 7f5d5be00640 1 -- 172.21.15.57:0/155798123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c1019c0 msgr2=0x7f5d5c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.807+0000 7f5d5be00640 1 --2- 172.21.15.57:0/155798123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c101e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009930 tx=0x7f5d4802a420 comp rx=0 tx=0).stop 2024-09-12T03:04:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 -- 172.21.15.57:0/155798123 shutdown_connections 2024-09-12T03:04:30.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 --2- 172.21.15.57:0/155798123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c102360 0x7f5d5c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 --2- 172.21.15.57:0/155798123 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c101e20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 --2- 172.21.15.57:0/155798123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c1079c0 0x7f5d5c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 -- 172.21.15.57:0/155798123 >> 172.21.15.57:0/155798123 conn(0x7f5d5c0fd740 msgr2=0x7f5d5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 -- 172.21.15.57:0/155798123 shutdown_connections 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.808+0000 7f5d5be00640 1 -- 172.21.15.57:0/155798123 wait complete. 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.809+0000 7f5d5be00640 1 Processor -- start 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.809+0000 7f5d5be00640 1 -- start start 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c102360 0x7f5d5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c1079c0 0x7f5d5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111e10 con 0x7f5d5c102360 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111c90 con 0x7f5d5c1019c0 2024-09-12T03:04:30.553 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5c111f90 con 0x7f5d5c1079c0 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c102360 0x7f5d5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c102360 0x7f5d5c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43640/0 (socket says 172.21.15.57:43640) 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5a000640 1 -- 172.21.15.57:0/2991797953 learned_addr learned my addr 172.21.15.57:0/2991797953 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58708/0 (socket says 172.21.15.57:58708) 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.810+0000 7f5d5b400640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c1079c0 0x7f5d5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 -- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c1079c0 msgr2=0x7f5d5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d5c1079c0 0x7f5d5c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 -- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c102360 msgr2=0x7f5d5c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d5c102360 0x7f5d5c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5c1a67d0 con 0x7f5d5c1019c0 2024-09-12T03:04:30.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d5aa00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c0744f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005fd0 tx=0x7f5d4c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c009c30 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.811+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d4c009dd0 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.812+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c010460 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.812+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5c1a6ac0 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.812+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5c1a6fb0 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.813+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d5c062900 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.816+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5d4c003f50 con 0x7f5d5c1019c0 2024-09-12T03:04:30.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d53400640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5d400a2c90 0x7f5d400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f5d4c0cf390 con 0x7f5d5c1019c0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d5a000640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5d400a2c90 0x7f5d400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d53400640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] conn(0x7f5d400ade60 0x7f5d400b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d4c00fec0 con 0x7f5d400ade60 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.818+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f5d4c0cf740 con 0x7f5d5c1019c0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.819+0000 7f5d5b400640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] conn(0x7f5d400ade60 0x7f5d400b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.819+0000 7f5d5a000640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5d400a2c90 0x7f5d400a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005c40 tx=0x7f5d48005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.819+0000 7f5d5b400640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] conn(0x7f5d400ade60 0x7f5d400b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.820+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== osd.11 v2:172.21.15.204:6824/1683140882 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d4c00fec0 con 0x7f5d400ade60 2024-09-12T03:04:30.556 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.835+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 --> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d5c062dd0 con 0x7f5d400ade60 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.835+0000 7f5d53400640 1 -- 172.21.15.57:0/2991797953 <== osd.11 v2:172.21.15.204:6824/1683140882 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d5c062dd0 con 0x7f5d400ade60 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] conn(0x7f5d400ade60 msgr2=0x7f5d400b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] conn(0x7f5d400ade60 0x7f5d400b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5d400a2c90 msgr2=0x7f5d400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5d400a2c90 0x7f5d400a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d48005c40 tx=0x7f5d48005b50 comp rx=0 tx=0).stop 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 msgr2=0x7f5d5c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 --2- 172.21.15.57:0/2991797953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d5c1019c0 0x7f5d5c0744f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c005fd0 tx=0x7f5d4c00dde0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.557 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5b400640 1 -- 172.21.15.57:0/2991797953 reap_dead start 2024-09-12T03:04:30.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 shutdown_connections 2024-09-12T03:04:30.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 >> 172.21.15.57:0/2991797953 conn(0x7f5d5c0fd740 msgr2=0x7f5d5c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 shutdown_connections 2024-09-12T03:04:30.558 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:29.836+0000 7f5d5be00640 1 -- 172.21.15.57:0/2991797953 wait complete. 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:31.504 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:31 smithi057 ceph-mon[30361]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 -- 172.21.15.57:0/3326421449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074590 msgr2=0x7f5e68074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 --2- 172.21.15.57:0/3326421449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074590 0x7f5e68074970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009ca0 tx=0x7f5e5c02a950 comp rx=0 tx=0).stop 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 -- 172.21.15.57:0/3326421449 shutdown_connections 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 --2- 172.21.15.57:0/3326421449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 0x7f5e68114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 --2- 172.21.15.57:0/3326421449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074eb0 0x7f5e681124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 --2- 172.21.15.57:0/3326421449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074590 0x7f5e68074970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 -- 172.21.15.57:0/3326421449 >> 172.21.15.57:0/3326421449 conn(0x7f5e680fd710 msgr2=0x7f5e680ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 -- 172.21.15.57:0/3326421449 shutdown_connections 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.239+0000 7f5e6e800640 1 -- 172.21.15.57:0/3326421449 wait complete. 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.240+0000 7f5e6e800640 1 Processor -- start 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 -- start start 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 0x7f5e681a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074eb0 0x7f5e681a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 0x7f5e6819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68118410 con 0x7f5e68074eb0 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68118290 con 0x7f5e681129e0 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68118590 con 0x7f5e68074590 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074eb0 0x7f5e681a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 0x7f5e6819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 0x7f5e681a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 0x7f5e6819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58742/0 (socket says 172.21.15.57:58742) 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 0x7f5e681a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:59958/0 (socket says 172.21.15.57:59958) 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.241+0000 7f5e6de00640 1 -- 172.21.15.57:0/846182555 learned_addr learned my addr 172.21.15.57:0/846182555 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 -- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 msgr2=0x7f5e6819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e681129e0 0x7f5e6819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 -- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074eb0 msgr2=0x7f5e681a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5e68074eb0 0x7f5e681a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6819efb0 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e6d400640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 0x7f5e681a4670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005e30 tx=0x7f5e5c003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.242+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c034040 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.243+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5c02f940 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.243+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0046e0 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.243+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6819f240 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.243+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681ab420 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.244+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e68062940 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.246+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e5c037020 con 0x7f5e68074590 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e66000640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5e500a2c90 0x7f5e500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f5e5c0f0080 con 0x7f5e68074590 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e6ca00640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5e500a2c90 0x7f5e500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e66000640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] conn(0x7f5e500ade60 0x7f5e500b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e5c02fc90 con 0x7f5e500ade60 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.247+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c0f0430 con 0x7f5e68074590 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.248+0000 7f5e6de00640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] conn(0x7f5e500ade60 0x7f5e500b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.248+0000 7f5e6ca00640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5e500a2c90 0x7f5e500a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e6819ff90 tx=0x7f5e54005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.249+0000 7f5e6de00640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] conn(0x7f5e500ade60 0x7f5e500b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.249+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== osd.1 v2:172.21.15.181:6800/1533372313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e5c02fc90 con 0x7f5e500ade60 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.264+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 --> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e68062d90 con 0x7f5e500ade60 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.264+0000 7f5e66000640 1 -- 172.21.15.57:0/846182555 <== osd.1 v2:172.21.15.181:6800/1533372313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e68062d90 con 0x7f5e500ade60 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] conn(0x7f5e500ade60 msgr2=0x7f5e500b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] conn(0x7f5e500ade60 0x7f5e500b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5e500a2c90 msgr2=0x7f5e500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5e500a2c90 0x7f5e500a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5e6819ff90 tx=0x7f5e54005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 msgr2=0x7f5e681a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 --2- 172.21.15.57:0/846182555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e68074590 0x7f5e681a4670 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005e30 tx=0x7f5e5c003c00 comp rx=0 tx=0).stop 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6de00640 1 -- 172.21.15.57:0/846182555 reap_dead start 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 shutdown_connections 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 >> 172.21.15.57:0/846182555 conn(0x7f5e680fd710 msgr2=0x7f5e680fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 shutdown_connections 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:30.265+0000 7f5e6e800640 1 -- 172.21.15.57:0/846182555 wait complete. 2024-09-12T03:04:31.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:31 smithi204 ceph-mon[35013]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:04:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:31 smithi181 ceph-mon[35320]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-12T03:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 -- 172.21.15.57:0/823002222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 msgr2=0x7f6ea0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 --2- 172.21.15.57:0/823002222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 0x7f6ea0110ca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e98009ca0 tx=0x7f6e9802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 -- 172.21.15.57:0/823002222 shutdown_connections 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 --2- 172.21.15.57:0/823002222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 0x7f6ea0110ca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 --2- 172.21.15.57:0/823002222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea01049a0 0x7f6ea01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 --2- 172.21.15.57:0/823002222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea0103ff0 0x7f6ea01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 -- 172.21.15.57:0/823002222 >> 172.21.15.57:0/823002222 conn(0x7f6ea00ffa90 msgr2=0x7f6ea0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 -- 172.21.15.57:0/823002222 shutdown_connections 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.123+0000 7f6ea6200640 1 -- 172.21.15.57:0/823002222 wait complete. 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.124+0000 7f6ea6200640 1 Processor -- start 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.124+0000 7f6ea6200640 1 -- start start 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.124+0000 7f6ea6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea0103ff0 0x7f6ea019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6ea6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 0x7f6ea019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6ea6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 0x7f6ea019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6ea6200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea01141a0 con 0x7f6ea0109120 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6ea6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea0114020 con 0x7f6ea0103ff0 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6ea6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea0114320 con 0x7f6ea01049a0 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e97e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 0x7f6ea019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea0103ff0 0x7f6ea019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e97e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 0x7f6ea019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:59980/0 (socket says 172.21.15.57:59980) 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 0x7f6ea019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e97e00640 1 -- 172.21.15.57:0/1310668587 learned_addr learned my addr 172.21.15.57:0/1310668587 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 -- 172.21.15.57:0/1310668587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 msgr2=0x7f6ea019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 0x7f6ea019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 -- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 msgr2=0x7f6ea019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ea0109120 0x7f6ea019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e9f400640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea019f590 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.125+0000 7f6e97e00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ea01049a0 0x7f6ea019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6e9f400640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea0103ff0 0x7f6ea019a5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9f0 tx=0x7f6e9000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000ccc0 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e90005020 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea019f880 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e900049e0 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.126+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea01a7160 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.127+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ea0101020 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.129+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6e90005560 con 0x7f6ea0103ff0 2024-09-12T03:04:33.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.130+0000 7f6e9cc00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6e840a2c10 0x7f6e840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6e900cf7a0 con 0x7f6ea0103ff0 2024-09-12T03:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e97e00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6e840a2c10 0x7f6e840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e9cc00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] conn(0x7f6e840adde0 0x7f6e840b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e840b0950 con 0x7f6e840adde0 2024-09-12T03:04:33.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f6e900cfb50 con 0x7f6ea0103ff0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e9fe00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] conn(0x7f6e840adde0 0x7f6e840b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.131+0000 7f6e97e00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6e840a2c10 0x7f6e840a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ea010ab00 tx=0x7f6e88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.132+0000 7f6e9fe00640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] conn(0x7f6e840adde0 0x7f6e840b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.133+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== osd.3 v2:172.21.15.57:6810/2741769998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e840b0950 con 0x7f6e840adde0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.147+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 --> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ea01054d0 con 0x7f6e840adde0 2024-09-12T03:04:33.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.147+0000 7f6e9cc00640 1 -- 172.21.15.57:0/1310668587 <== osd.3 v2:172.21.15.57:6810/2741769998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ea01054d0 con 0x7f6e840adde0 2024-09-12T03:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] conn(0x7f6e840adde0 msgr2=0x7f6e840b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] conn(0x7f6e840adde0 0x7f6e840b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6e840a2c10 msgr2=0x7f6e840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6e840a2c10 0x7f6e840a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ea010ab00 tx=0x7f6e88005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea0103ff0 msgr2=0x7f6ea019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 --2- 172.21.15.57:0/1310668587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ea0103ff0 0x7f6ea019a5e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000e9f0 tx=0x7f6e9000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6e9fe00640 1 -- 172.21.15.57:0/1310668587 reap_dead start 2024-09-12T03:04:33.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 shutdown_connections 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 >> 172.21.15.57:0/1310668587 conn(0x7f6ea00ffa90 msgr2=0x7f6ea01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 shutdown_connections 2024-09-12T03:04:33.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:32.148+0000 7f6ea6200640 1 -- 172.21.15.57:0/1310668587 wait complete. 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 -- 172.21.15.57:0/756573597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c041049c0 msgr2=0x7f0c04108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 --2- 172.21.15.57:0/756573597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c041049c0 0x7f0c04108a10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009900 tx=0x7f0bf002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 -- 172.21.15.57:0/756573597 shutdown_connections 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 --2- 172.21.15.57:0/756573597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c04109140 0x7f0c04110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 --2- 172.21.15.57:0/756573597 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c041049c0 0x7f0c04108a10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 --2- 172.21.15.57:0/756573597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c04104010 0x7f0c041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.471+0000 7f0c0c000640 1 -- 172.21.15.57:0/756573597 >> 172.21.15.57:0/756573597 conn(0x7f0c040ffa50 msgr2=0x7f0c04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.472+0000 7f0c0c000640 1 -- 172.21.15.57:0/756573597 shutdown_connections 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.472+0000 7f0c0c000640 1 -- 172.21.15.57:0/756573597 wait complete. 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.472+0000 7f0c0c000640 1 Processor -- start 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 -- start start 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 0x7f0c04076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c041049c0 0x7f0c04076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c04109140 0x7f0c04077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04114140 con 0x7f0c04104010 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04113fc0 con 0x7f0c04109140 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041142c0 con 0x7f0c041049c0 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c09600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 0x7f0c04076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c09600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 0x7f0c04076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43730/0 (socket says 172.21.15.57:43730) 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c09600640 1 -- 172.21.15.57:0/1814635424 learned_addr learned my addr 172.21.15.57:0/1814635424 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c0a000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c04109140 0x7f0c04077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.473+0000 7f0c08c00640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c041049c0 0x7f0c04076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 -- 172.21.15.57:0/1814635424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c041049c0 msgr2=0x7f0c04076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c041049c0 0x7f0c04076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 -- 172.21.15.57:0/1814635424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c04109140 msgr2=0x7f0c04077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c04109140 0x7f0c04077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041aad00 con 0x7f0c04104010 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c09600640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 0x7f0c04076700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800d980 tx=0x7f0bf800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8017070 con 0x7f0c04104010 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf8009c30 con 0x7f0c04104010 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800f6b0 con 0x7f0c04104010 2024-09-12T03:04:33.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041aaff0 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.474+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041ab450 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.475+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c04062980 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.477+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0bf8010460 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c02000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0bec0a2c90 0x7f0bec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f0bf80cf410 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c08c00640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0bec0a2c90 0x7f0bec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c02000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] conn(0x7f0bec0ade60 0x7f0bec0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bf8016ec0 con 0x7f0bec0ade60 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f0bf80cf7c0 con 0x7f0c04104010 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.479+0000 7f0c0a000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] conn(0x7f0bec0ade60 0x7f0bec0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.480+0000 7f0c08c00640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0bec0a2c90 0x7f0bec0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009230 tx=0x7f0bf00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.480+0000 7f0c0a000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] conn(0x7f0bec0ade60 0x7f0bec0b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:33.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.481+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== osd.0 v2:172.21.15.57:6802/3978791581 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bf8016ec0 con 0x7f0bec0ade60 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.495+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 --> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c04062e00 con 0x7f0bec0ade60 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c02000640 1 -- 172.21.15.57:0/1814635424 <== osd.0 v2:172.21.15.57:6802/3978791581 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c04062e00 con 0x7f0bec0ade60 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] conn(0x7f0bec0ade60 msgr2=0x7f0bec0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] conn(0x7f0bec0ade60 0x7f0bec0b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0bec0a2c90 msgr2=0x7f0bec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f0bec0a2c90 0x7f0bec0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009230 tx=0x7f0bf00058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:33.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 msgr2=0x7f0c04076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 --2- 172.21.15.57:0/1814635424 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0c04104010 0x7f0c04076700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800d980 tx=0x7f0bf800de40 comp rx=0 tx=0).stop 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0a000640 1 -- 172.21.15.57:0/1814635424 reap_dead start 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 shutdown_connections 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.496+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 >> 172.21.15.57:0/1814635424 conn(0x7f0c040ffa50 msgr2=0x7f0c04100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.497+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 shutdown_connections 2024-09-12T03:04:33.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:33.497+0000 7f0c0c000640 1 -- 172.21.15.57:0/1814635424 wait complete. 2024-09-12T03:04:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:33 smithi181 ceph-mon[35320]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:04:33.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:33 smithi057 ceph-mon[30361]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:04:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:33 smithi204 ceph-mon[35013]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-12T03:04:34.905 INFO:teuthology.orchestra.run.smithi057.stdout:137438953540 2024-09-12T03:04:34.905 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:04:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 -- 172.21.15.57:0/3208267039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 msgr2=0x7f8700100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 --2- 172.21.15.57:0/3208267039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f8700100f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002a410 comp rx=0 tx=0).stop 2024-09-12T03:04:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 -- 172.21.15.57:0/3208267039 shutdown_connections 2024-09-12T03:04:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 --2- 172.21.15.57:0/3208267039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f870010e500 0x7f8700110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.167 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 --2- 172.21.15.57:0/3208267039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8700101440 0x7f870010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 --2- 172.21.15.57:0/3208267039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f8700100f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 -- 172.21.15.57:0/3208267039 >> 172.21.15.57:0/3208267039 conn(0x7f87000fc7d0 msgr2=0x7f87000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 -- 172.21.15.57:0/3208267039 shutdown_connections 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.083+0000 7f8704a00640 1 -- 172.21.15.57:0/3208267039 wait complete. 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.084+0000 7f8704a00640 1 Processor -- start 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.084+0000 7f8704a00640 1 -- start start 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f87001a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700101440 0x7f87001a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f870010e500 0x7f870019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f87001142b0 con 0x7f870010e500 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8700114130 con 0x7f8700101440 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f8704a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8700114430 con 0x7f8700100b20 2024-09-12T03:04:35.168 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f86fde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f87001a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f86fe800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f870010e500 0x7f870019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f86fd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700101440 0x7f87001a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f86fde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f87001a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60028/0 (socket says 172.21.15.57:60028) 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.085+0000 7f86fde00640 1 -- 172.21.15.57:0/338590099 learned_addr learned my addr 172.21.15.57:0/338590099 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 -- 172.21.15.57:0/338590099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 msgr2=0x7f87001a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8700100b20 0x7f87001a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 -- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f870010e500 msgr2=0x7f870019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f870010e500 0x7f870019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870019acf0 con 0x7f8700101440 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fe800640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f870010e500 0x7f870019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86fd400640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700101440 0x7f87001a0940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9c0 tx=0x7f86ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00ccc0 con 0x7f8700101440 2024-09-12T03:04:35.169 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870019afe0 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec0035f0 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00ce60 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.086+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87001a72d0 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.087+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8700062940 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.090+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f86ec0037a0 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86f6a00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86e40a2c90 0x7f86e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f86ec0cfbf0 con 0x7f8700101440 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86fde00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86e40a2c90 0x7f86e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86f6a00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] conn(0x7f86e40ade60 0x7f86e40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.170 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86ec017e50 con 0x7f86e40ade60 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f86ec0cffa0 con 0x7f8700101440 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.091+0000 7f86fe800640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] conn(0x7f86e40ade60 0x7f86e40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.092+0000 7f86fde00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86e40a2c90 0x7f86e40a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86f00096c0 tx=0x7f86f00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.092+0000 7f86fe800640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] conn(0x7f86e40ade60 0x7f86e40b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.093+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== osd.9 v2:172.21.15.57:6826/3989705460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86ec017e50 con 0x7f86e40ade60 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.107+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 --> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8700062d40 con 0x7f86e40ade60 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f86f6a00640 1 -- 172.21.15.57:0/338590099 <== osd.9 v2:172.21.15.57:6826/3989705460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8700062d40 con 0x7f86e40ade60 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] conn(0x7f86e40ade60 msgr2=0x7f86e40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] conn(0x7f86e40ade60 0x7f86e40b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86e40a2c90 msgr2=0x7f86e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86e40a2c90 0x7f86e40a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f86f00096c0 tx=0x7f86f00046c0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700101440 msgr2=0x7f87001a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f8704a00640 1 --2- 172.21.15.57:0/338590099 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700101440 0x7f87001a0940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9c0 tx=0x7f86ec00ee80 comp rx=0 tx=0).stop 2024-09-12T03:04:35.171 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.108+0000 7f86fe800640 1 -- 172.21.15.57:0/338590099 reap_dead start 2024-09-12T03:04:35.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.109+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 shutdown_connections 2024-09-12T03:04:35.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.109+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 >> 172.21.15.57:0/338590099 conn(0x7f87000fc7d0 msgr2=0x7f87000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:35.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.109+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 shutdown_connections 2024-09-12T03:04:35.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.109+0000 7f8704a00640 1 -- 172.21.15.57:0/338590099 wait complete. 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 -- 172.21.15.57:0/1392106052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4102790 msgr2=0x7fd3e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 --2- 172.21.15.57:0/1392106052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4102790 0x7fd3e41067e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009900 tx=0x7fd3d002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 -- 172.21.15.57:0/1392106052 shutdown_connections 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 --2- 172.21.15.57:0/1392106052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4106f10 0x7fd3e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 --2- 172.21.15.57:0/1392106052 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4102790 0x7fd3e41067e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 --2- 172.21.15.57:0/1392106052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4101de0 0x7fd3e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 -- 172.21.15.57:0/1392106052 >> 172.21.15.57:0/1392106052 conn(0x7fd3e40fd8c0 msgr2=0x7fd3e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 -- 172.21.15.57:0/1392106052 shutdown_connections 2024-09-12T03:04:35.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.567+0000 7fd3ec000640 1 -- 172.21.15.57:0/1392106052 wait complete. 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 Processor -- start 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 -- start start 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4101de0 0x7fd3e419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 0x7fd3e419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4106f10 0x7fd3e419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111f80 con 0x7fd3e4101de0 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4111e00 con 0x7fd3e4106f10 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.568+0000 7fd3ec000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e4112100 con 0x7fd3e4102790 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 0x7fd3e419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e9600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4101de0 0x7fd3e419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 0x7fd3e419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60050/0 (socket says 172.21.15.57:60050) 2024-09-12T03:04:35.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 -- 172.21.15.57:0/3092172531 learned_addr learned my addr 172.21.15.57:0/3092172531 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3ea000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4106f10 0x7fd3e419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 -- 172.21.15.57:0/3092172531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4106f10 msgr2=0x7fd3e419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e4106f10 0x7fd3e419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 -- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4101de0 msgr2=0x7fd3e419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4101de0 0x7fd3e419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e8c00640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e419f640 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.569+0000 7fd3e9600640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd3e4101de0 0x7fd3e419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3e8c00640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 0x7fd3e419abd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0004770 tx=0x7fd3d0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d003f070 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e419f8d0 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0031ca0 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0035030 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.570+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e41a7210 con 0x7fd3e4102790 2024-09-12T03:04:35.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.571+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3e4075780 con 0x7fd3e4102790 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.573+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd3d0042070 con 0x7fd3e4102790 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e2000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd3cc0a2c10 0x7fd3cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fd3d00efd80 con 0x7fd3e4102790 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e9600640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd3cc0a2c10 0x7fd3cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e2000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] conn(0x7fd3cc0addd0 0x7fd3cc0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3d0046e50 con 0x7fd3cc0addd0 2024-09-12T03:04:35.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.575+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd3d00f0130 con 0x7fd3e4102790 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.576+0000 7fd3ea000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] conn(0x7fd3cc0addd0 0x7fd3cc0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.576+0000 7fd3e9600640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd3cc0a2c10 0x7fd3cc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009fc0 tx=0x7fd3d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.576+0000 7fd3ea000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] conn(0x7fd3cc0addd0 0x7fd3cc0b0270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.577+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== osd.5 v2:172.21.15.204:6808/4265194812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3d0046e50 con 0x7fd3cc0addd0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.592+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 --> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3e41032c0 con 0x7fd3cc0addd0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3e2000640 1 -- 172.21.15.57:0/3092172531 <== osd.5 v2:172.21.15.204:6808/4265194812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd3e41032c0 con 0x7fd3cc0addd0 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] conn(0x7fd3cc0addd0 msgr2=0x7fd3cc0b0270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] conn(0x7fd3cc0addd0 0x7fd3cc0b0270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd3cc0a2c10 msgr2=0x7fd3cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fd3cc0a2c10 0x7fd3cc0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8009fc0 tx=0x7fd3d8005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 msgr2=0x7fd3e419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 --2- 172.21.15.57:0/3092172531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3e4102790 0x7fd3e419abd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0004770 tx=0x7fd3d0003c10 comp rx=0 tx=0).stop 2024-09-12T03:04:35.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ea000640 1 -- 172.21.15.57:0/3092172531 reap_dead start 2024-09-12T03:04:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 shutdown_connections 2024-09-12T03:04:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 >> 172.21.15.57:0/3092172531 conn(0x7fd3e40fd8c0 msgr2=0x7fd3e40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 shutdown_connections 2024-09-12T03:04:35.600 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:35.593+0000 7fd3ec000640 1 -- 172.21.15.57:0/3092172531 wait complete. 2024-09-12T03:04:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:35 smithi181 ceph-mon[35320]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:35 smithi057 ceph-mon[30361]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:35 smithi204 ceph-mon[35013]: pgmap v351: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:36.055 INFO:teuthology.orchestra.run.smithi057.stdout:34359738458 2024-09-12T03:04:36.055 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:04:36.540 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:36.574 INFO:teuthology.orchestra.run.smithi057.stdout:330712481823 2024-09-12T03:04:36.574 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:04:37.311 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 -- 172.21.15.57:0/488123606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc1047e0 msgr2=0x7f85cc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 --2- 172.21.15.57:0/488123606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc1047e0 0x7f85cc108830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009900 tx=0x7f85b802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 -- 172.21.15.57:0/488123606 shutdown_connections 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 --2- 172.21.15.57:0/488123606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc108f60 0x7f85cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 --2- 172.21.15.57:0/488123606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc1047e0 0x7f85cc108830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 --2- 172.21.15.57:0/488123606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85cc103e30 0x7f85cc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 -- 172.21.15.57:0/488123606 >> 172.21.15.57:0/488123606 conn(0x7f85cc0ff910 msgr2=0x7f85cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 -- 172.21.15.57:0/488123606 shutdown_connections 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.280+0000 7f85d3000640 1 -- 172.21.15.57:0/488123606 wait complete. 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.281+0000 7f85d3000640 1 Processor -- start 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.281+0000 7f85d3000640 1 -- start start 2024-09-12T03:04:37.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc103e30 0x7f85cc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc1047e0 0x7f85cc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85cc108f60 0x7f85cc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc113fd0 con 0x7f85cc103e30 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc113e50 con 0x7f85cc1047e0 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc114150 con 0x7f85cc108f60 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d1c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc103e30 0x7f85cc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc1047e0 0x7f85cc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d1c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc103e30 0x7f85cc19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43760/0 (socket says 172.21.15.57:43760) 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d1c00640 1 -- 172.21.15.57:0/1727875749 learned_addr learned my addr 172.21.15.57:0/1727875749 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.282+0000 7f85d2600640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85cc108f60 0x7f85cc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 -- 172.21.15.57:0/1727875749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85cc108f60 msgr2=0x7f85cc19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85cc108f60 0x7f85cc19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 -- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc103e30 msgr2=0x7f85cc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f85cc103e30 0x7f85cc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc19f4c0 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85d1200640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc1047e0 0x7f85cc19aa50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85b802a910 tx=0x7f85b8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.283+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b803f070 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.284+0000 7f85d3000640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc19f750 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.284+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b8031c90 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.284+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8035030 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.284+0000 7f85d3000640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc1a7090 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.285+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85cc062900 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.287+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85b8042070 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.289+0000 7f85ca000640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85b40a2c90 0x7f85b40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.289+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f85b80f04c0 con 0x7f85cc1047e0 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.290+0000 7f85ca000640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] conn(0x7f85b40ade60 0x7f85b40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:37.314 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.290+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85b800b0b0 con 0x7f85b40ade60 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.290+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f85b80f0870 con 0x7f85cc1047e0 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.291+0000 7f85d1c00640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85b40a2c90 0x7f85b40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.292+0000 7f85d1c00640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85b40a2c90 0x7f85b40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.293+0000 7f85d2600640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] conn(0x7f85b40ade60 0x7f85b40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.294+0000 7f85d2600640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] conn(0x7f85b40ade60 0x7f85b40b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.295+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== osd.10 v2:172.21.15.181:6824/25222354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85b800b0b0 con 0x7f85b40ade60 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.310+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 --> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85cc062d00 con 0x7f85b40ade60 2024-09-12T03:04:37.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85ca000640 1 -- 172.21.15.57:0/1727875749 <== osd.10 v2:172.21.15.181:6824/25222354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f85cc062d00 con 0x7f85b40ade60 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] conn(0x7f85b40ade60 msgr2=0x7f85b40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] conn(0x7f85b40ade60 0x7f85b40b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85b40a2c90 msgr2=0x7f85b40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f85b40a2c90 0x7f85b40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c000b480 comp rx=0 tx=0).stop 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc1047e0 msgr2=0x7f85cc19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 --2- 172.21.15.57:0/1727875749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85cc1047e0 0x7f85cc19aa50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85b802a910 tx=0x7f85b8003b90 comp rx=0 tx=0).stop 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85d2600640 1 -- 172.21.15.57:0/1727875749 reap_dead start 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 shutdown_connections 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 >> 172.21.15.57:0/1727875749 conn(0x7f85cc0ff910 msgr2=0x7f85cc100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 shutdown_connections 2024-09-12T03:04:37.316 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:36.311+0000 7f85bf400640 1 -- 172.21.15.57:0/1727875749 wait complete. 2024-09-12T03:04:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:37 smithi181 ceph-mon[35320]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:37.785 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:37 smithi057 ceph-mon[30361]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:37 smithi204 ceph-mon[35013]: pgmap v352: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:38.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.135+0000 7fb99f200640 1 -- 172.21.15.57:0/2793822704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 msgr2=0x7fb9a0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.135+0000 7fb99f200640 1 --2- 172.21.15.57:0/2793822704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a0074970 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb990009900 tx=0x7fb99002a480 comp rx=0 tx=0).stop 2024-09-12T03:04:38.258 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 -- 172.21.15.57:0/2793822704 shutdown_connections 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 --2- 172.21.15.57:0/2793822704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a01129e0 0x7fb9a0114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 --2- 172.21.15.57:0/2793822704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a0074eb0 0x7fb9a01124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 --2- 172.21.15.57:0/2793822704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a0074970 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 -- 172.21.15.57:0/2793822704 >> 172.21.15.57:0/2793822704 conn(0x7fb9a00fd710 msgr2=0x7fb9a00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 -- 172.21.15.57:0/2793822704 shutdown_connections 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.136+0000 7fb99f200640 1 -- 172.21.15.57:0/2793822704 wait complete. 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 Processor -- start 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 -- start start 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a0074eb0 0x7fb9a01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a01129e0 0x7fb9a019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0118410 con 0x7fb9a0074590 2024-09-12T03:04:38.259 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0118290 con 0x7fb9a0074eb0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0118590 con 0x7fb9a01129e0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.137+0000 7fb99de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a01a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56938/0 (socket says 172.21.15.57:56938) 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99de00640 1 -- 172.21.15.57:0/1789378813 learned_addr learned my addr 172.21.15.57:0/1789378813 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a0074eb0 0x7fb9a01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99e800640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a01129e0 0x7fb9a019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 -- 172.21.15.57:0/1789378813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a01129e0 msgr2=0x7fb9a019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9a01129e0 0x7fb9a019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 -- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 msgr2=0x7fb9a01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9a0074590 0x7fb9a01a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a019efb0 con 0x7fb9a0074eb0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb99d400640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a0074eb0 0x7fb9a01a4bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00e9f0 tx=0x7fb98c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.260 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.138+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00ccc0 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.139+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb98c005020 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.139+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0049e0 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.139+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a019f2a0 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.139+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01ab420 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.139+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9a0062940 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.142+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb98c005560 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.143+0000 7fb996a00640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb9840a2c90 0x7fb9840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.143+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb98c0cf750 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.143+0000 7fb99de00640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb9840a2c90 0x7fb9840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.143+0000 7fb996a00640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] conn(0x7fb9840ade60 0x7fb9840b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.144+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9840b09d0 con 0x7fb9840ade60 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.144+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb98c0cfb00 con 0x7fb9a0074eb0 2024-09-12T03:04:38.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.144+0000 7fb99e800640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] conn(0x7fb9840ade60 0x7fb9840b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.144+0000 7fb99de00640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb9840a2c90 0x7fb9840a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb9900096c0 tx=0x7fb9900046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.144+0000 7fb99e800640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] conn(0x7fb9840ade60 0x7fb9840b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.145+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== osd.6 v2:172.21.15.57:6818/1487516632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9840b09d0 con 0x7fb9840ade60 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.161+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 --> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9a0062dc0 con 0x7fb9840ade60 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.161+0000 7fb996a00640 1 -- 172.21.15.57:0/1789378813 <== osd.6 v2:172.21.15.57:6818/1487516632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9a0062dc0 con 0x7fb9840ade60 2024-09-12T03:04:38.262 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] conn(0x7fb9840ade60 msgr2=0x7fb9840b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] conn(0x7fb9840ade60 0x7fb9840b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb9840a2c90 msgr2=0x7fb9840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb9840a2c90 0x7fb9840a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb9900096c0 tx=0x7fb9900046c0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a0074eb0 msgr2=0x7fb9a01a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 --2- 172.21.15.57:0/1789378813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9a0074eb0 0x7fb9a01a4bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00e9f0 tx=0x7fb98c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99e800640 1 -- 172.21.15.57:0/1789378813 reap_dead start 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 shutdown_connections 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 >> 172.21.15.57:0/1789378813 conn(0x7fb9a00fd710 msgr2=0x7fb9a00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 shutdown_connections 2024-09-12T03:04:38.263 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.162+0000 7fb99f200640 1 -- 172.21.15.57:0/1789378813 wait complete. 2024-09-12T03:04:38.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 -- 172.21.15.57:0/4036364616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc103e30 msgr2=0x7f27fc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 --2- 172.21.15.57:0/4036364616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc103e30 0x7f27fc104210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009900 tx=0x7f27f002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 -- 172.21.15.57:0/4036364616 shutdown_connections 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 --2- 172.21.15.57:0/4036364616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc108f60 0x7f27fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 --2- 172.21.15.57:0/4036364616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc1047e0 0x7f27fc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 --2- 172.21.15.57:0/4036364616 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc103e30 0x7f27fc104210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 -- 172.21.15.57:0/4036364616 >> 172.21.15.57:0/4036364616 conn(0x7f27fc0ff910 msgr2=0x7f27fc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 -- 172.21.15.57:0/4036364616 shutdown_connections 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.692+0000 7f2802200640 1 -- 172.21.15.57:0/4036364616 wait complete. 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.693+0000 7f2802200640 1 Processor -- start 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.693+0000 7f2802200640 1 -- start start 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.693+0000 7f2802200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc103e30 0x7f27fc19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.737 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2802200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc1047e0 0x7f27fc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2802200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 0x7f27fc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2802200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc113fd0 con 0x7f27fc108f60 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2802200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc113e50 con 0x7f27fc103e30 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2802200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc114150 con 0x7f27fc1047e0 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 0x7f27fc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 0x7f27fc19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:56988/0 (socket says 172.21.15.57:56988) 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 -- 172.21.15.57:0/219145782 learned_addr learned my addr 172.21.15.57:0/219145782 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2800e00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc103e30 0x7f27fc19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f27fbe00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc1047e0 0x7f27fc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 -- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc1047e0 msgr2=0x7f27fc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27fc1047e0 0x7f27fc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 -- 172.21.15.57:0/219145782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc103e30 msgr2=0x7f27fc19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc103e30 0x7f27fc19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2801800640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc19f4c0 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.694+0000 7f2800e00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27fc103e30 0x7f27fc19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f2801800640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 0x7f27fc19ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27f400eb10 tx=0x7f27f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004b20 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f4004cc0 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4010760 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc19f7b0 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.695+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1a7090 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.696+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27fc062900 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.698+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27f4005020 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.699+0000 7f27f9600640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f27e40a2c90 0x7f27e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.699+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f27f40cfe30 con 0x7f27fc108f60 2024-09-12T03:04:38.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f2800e00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f27e40a2c90 0x7f27e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f27f9600640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] conn(0x7f27e40ade60 0x7f27e40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27e40b09d0 con 0x7f27e40ade60 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f27f40d01e0 con 0x7f27fc108f60 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f27fbe00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] conn(0x7f27e40ade60 0x7f27e40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.700+0000 7f2800e00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f27e40a2c90 0x7f27e40a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27f0000c00 tx=0x7f27f0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.701+0000 7f27fbe00640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] conn(0x7f27e40ade60 0x7f27e40b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.702+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== osd.2 v2:172.21.15.204:6800/82472683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27e40b09d0 con 0x7f27e40ade60 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.716+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 --> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27fc062d80 con 0x7f27e40ade60 2024-09-12T03:04:38.740 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.717+0000 7f27f9600640 1 -- 172.21.15.57:0/219145782 <== osd.2 v2:172.21.15.204:6800/82472683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f27fc062d80 con 0x7f27e40ade60 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] conn(0x7f27e40ade60 msgr2=0x7f27e40b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] conn(0x7f27e40ade60 0x7f27e40b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f27e40a2c90 msgr2=0x7f27e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f27e40a2c90 0x7f27e40a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27f0000c00 tx=0x7f27f0031510 comp rx=0 tx=0).stop 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 msgr2=0x7f27fc19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 --2- 172.21.15.57:0/219145782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f27fc108f60 0x7f27fc19ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27f400eb10 tx=0x7f27f400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2801800640 1 -- 172.21.15.57:0/219145782 reap_dead start 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 shutdown_connections 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 >> 172.21.15.57:0/219145782 conn(0x7f27fc0ff910 msgr2=0x7f27fc100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:38.741 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 shutdown_connections 2024-09-12T03:04:38.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.718+0000 7f2802200640 1 -- 172.21.15.57:0/219145782 wait complete. 2024-09-12T03:04:39.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.289+0000 7f573fe00640 1 -- 172.21.15.57:0/2515754606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740076720 msgr2=0x7f5740076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.289+0000 7f573fe00640 1 --2- 172.21.15.57:0/2515754606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740076720 0x7f5740076b00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5730009900 tx=0x7f573002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 -- 172.21.15.57:0/2515754606 shutdown_connections 2024-09-12T03:04:39.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 --2- 172.21.15.57:0/2515754606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740112870 0x7f5740114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.092 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 --2- 172.21.15.57:0/2515754606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5740077040 0x7f5740112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 --2- 172.21.15.57:0/2515754606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740076720 0x7f5740076b00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 -- 172.21.15.57:0/2515754606 >> 172.21.15.57:0/2515754606 conn(0x7f57400ff8d0 msgr2=0x7f5740101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 -- 172.21.15.57:0/2515754606 shutdown_connections 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.290+0000 7f573fe00640 1 -- 172.21.15.57:0/2515754606 wait complete. 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 Processor -- start 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 -- start start 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740076720 0x7f57401a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740077040 0x7f57401a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5740112870 0x7f574019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5740118430 con 0x7f5740112870 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f57401182b0 con 0x7f5740077040 2024-09-12T03:04:39.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57401185b0 con 0x7f5740076720 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5740112870 0x7f574019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740076720 0x7f57401a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f5737e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740077040 0x7f57401a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740076720 0x7f57401a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56446/0 (socket says 172.21.15.57:56446) 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.291+0000 7f573ea00640 1 -- 172.21.15.57:0/2306860548 learned_addr learned my addr 172.21.15.57:0/2306860548 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 -- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740076720 msgr2=0x7f57401a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740076720 0x7f57401a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 -- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5740112870 msgr2=0x7f574019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5740112870 0x7f574019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574019ef80 con 0x7f5740077040 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737e00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740077040 0x7f57401a4b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.292+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572800ccc0 con 0x7f5740077040 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.293+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574019f270 con 0x7f5740077040 2024-09-12T03:04:39.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.293+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728010910 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.293+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728018960 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.293+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57401ab5c0 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.293+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5740062940 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.296+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5728010ac0 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.297+0000 7f5737400640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f57240a2c90 0x7f57240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.297+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f57280d00b0 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f573ea00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f57240a2c90 0x7f57240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f5737400640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] conn(0x7f57240ade60 0x7f57240b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5728028ea0 con 0x7f57240ade60 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f57280d0460 con 0x7f5740077040 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f573f400640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] conn(0x7f57240ade60 0x7f57240b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.298+0000 7f573ea00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f57240a2c90 0x7f57240a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57300096c0 tx=0x7f57300046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.095 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.299+0000 7f573f400640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] conn(0x7f57240ade60 0x7f57240b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.300+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== osd.8 v2:172.21.15.204:6816/2786968053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5728028ea0 con 0x7f57240ade60 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.314+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 --> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5740062d40 con 0x7f57240ade60 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f5737400640 1 -- 172.21.15.57:0/2306860548 <== osd.8 v2:172.21.15.204:6816/2786968053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5740062d40 con 0x7f57240ade60 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] conn(0x7f57240ade60 msgr2=0x7f57240b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] conn(0x7f57240ade60 0x7f57240b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f57240a2c90 msgr2=0x7f57240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f57240a2c90 0x7f57240a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57300096c0 tx=0x7f57300046c0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.096 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740077040 msgr2=0x7f57401a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 --2- 172.21.15.57:0/2306860548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740077040 0x7f57401a4b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f572800e9f0 tx=0x7f572800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573f400640 1 -- 172.21.15.57:0/2306860548 reap_dead start 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 shutdown_connections 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 >> 172.21.15.57:0/2306860548 conn(0x7f57400ff8d0 msgr2=0x7f5740101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 shutdown_connections 2024-09-12T03:04:39.097 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:38.315+0000 7f573fe00640 1 -- 172.21.15.57:0/2306860548 wait complete. 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 -- 172.21.15.57:0/2614564459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2468102360 msgr2=0x7f246810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 --2- 172.21.15.57:0/2614564459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2468102360 0x7f246810e890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2460009cd0 tx=0x7f246002a8f0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 -- 172.21.15.57:0/2614564459 shutdown_connections 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 --2- 172.21.15.57:0/2614564459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2468102360 0x7f246810e890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 --2- 172.21.15.57:0/2614564459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681019c0 0x7f2468101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 --2- 172.21.15.57:0/2614564459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f24681079c0 0x7f2468107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.553+0000 7f2467200640 1 -- 172.21.15.57:0/2614564459 >> 172.21.15.57:0/2614564459 conn(0x7f24680fd740 msgr2=0x7f24680ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.554+0000 7f2467200640 1 -- 172.21.15.57:0/2614564459 shutdown_connections 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.554+0000 7f2467200640 1 -- 172.21.15.57:0/2614564459 wait complete. 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.554+0000 7f2467200640 1 Processor -- start 2024-09-12T03:04:39.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 -- start start 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24681019c0 0x7f246819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2468102360 0x7f246819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681079c0 0x7f246819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2468111e10 con 0x7f2468102360 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2468111c90 con 0x7f24681079c0 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2467200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2468111f90 con 0x7f24681019c0 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2465400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2468102360 0x7f246819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2465400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2468102360 0x7f246819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57016/0 (socket says 172.21.15.57:57016) 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2466800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681079c0 0x7f246819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2465400640 1 -- 172.21.15.57:0/3003752931 learned_addr learned my addr 172.21.15.57:0/3003752931 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.555+0000 7f2465e00640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24681019c0 0x7f246819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 -- 172.21.15.57:0/3003752931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24681019c0 msgr2=0x7f246819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24681019c0 0x7f246819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 -- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2468102360 msgr2=0x7f246819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2468102360 0x7f246819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246819f460 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f2466800640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681079c0 0x7f246819ed80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f246002ae10 tx=0x7f2460003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.556+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460030e40 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.557+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246819f6f0 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.557+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2460040070 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.557+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460045a30 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.557+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24681a7030 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.558+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f246810df00 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.560+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2460045bd0 con 0x7f24681079c0 2024-09-12T03:04:39.665 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.561+0000 7f245ea00640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f244c0a2c10 0x7f244c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.561+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f24600f0790 con 0x7f24681079c0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f2465e00640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f244c0a2c10 0x7f244c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f245ea00640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] conn(0x7f244c0adde0 0x7f244c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f246003ce50 con 0x7f244c0adde0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f24600f0b40 con 0x7f24681079c0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f2465400640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] conn(0x7f244c0adde0 0x7f244c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.562+0000 7f2465e00640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f244c0a2c10 0x7f244c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2458009b50 tx=0x7f2458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.563+0000 7f2465400640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] conn(0x7f244c0adde0 0x7f244c0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.564+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== osd.7 v2:172.21.15.181:6816/159674330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f246003ce50 con 0x7f244c0adde0 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.578+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 --> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2468103100 con 0x7f244c0adde0 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f245ea00640 1 -- 172.21.15.57:0/3003752931 <== osd.7 v2:172.21.15.181:6816/159674330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2468103100 con 0x7f244c0adde0 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] conn(0x7f244c0adde0 msgr2=0x7f244c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] conn(0x7f244c0adde0 0x7f244c0b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f244c0a2c10 msgr2=0x7f244c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f244c0a2c10 0x7f244c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2458009b50 tx=0x7f2458005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681079c0 msgr2=0x7f246819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 --2- 172.21.15.57:0/3003752931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24681079c0 0x7f246819ed80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f246002ae10 tx=0x7f2460003f10 comp rx=0 tx=0).stop 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2466800640 1 -- 172.21.15.57:0/3003752931 reap_dead start 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 shutdown_connections 2024-09-12T03:04:39.667 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 >> 172.21.15.57:0/3003752931 conn(0x7f24680fd740 msgr2=0x7f246810d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:39.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 shutdown_connections 2024-09-12T03:04:39.668 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:39.579+0000 7f2467200640 1 -- 172.21.15.57:0/3003752931 wait complete. 2024-09-12T03:04:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:39 smithi181 ceph-mon[35320]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:04:39.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:39 smithi057 ceph-mon[30361]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:04:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:39 smithi204 ceph-mon[35013]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T03:04:40.385 INFO:teuthology.orchestra.run.smithi057.stdout:244813135920 2024-09-12T03:04:40.386 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:04:41.145 INFO:teuthology.orchestra.run.smithi057.stdout:279172874281 2024-09-12T03:04:41.145 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:04:41.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:41 smithi057 ceph-mon[30361]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:04:41.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-12T03:04:41.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-12T03:04:41.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-12T03:04:41.695 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:41 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:04:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:41 smithi181 ceph-mon[35320]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:04:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-12T03:04:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-12T03:04:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-12T03:04:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:41 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:04:41.843 INFO:teuthology.orchestra.run.smithi057.stdout:214748364853 2024-09-12T03:04:41.844 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:04:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:41 smithi204 ceph-mon[35013]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T03:04:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-09-12T03:04:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-12T03:04:41.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-09-12T03:04:41.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:41 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T03:04:42.200 INFO:teuthology.orchestra.run.smithi057.stdout:55834574933 2024-09-12T03:04:42.201 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:04:42.756 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-12T03:04:42.756 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-12T03:04:42.756 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-12T03:04:42.757 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:42 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:04:42.757 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:42 smithi057 ceph-mon[30361]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:04:42.794 INFO:teuthology.orchestra.run.smithi057.stdout:158913790016 2024-09-12T03:04:42.795 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:04:42.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-12T03:04:42.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-12T03:04:42.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-12T03:04:42.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:42 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:04:42.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:42 smithi204 ceph-mon[35013]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:04:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-09-12T03:04:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-12T03:04:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-09-12T03:04:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:42 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T03:04:42.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:42 smithi181 ceph-mon[35320]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T03:04:43.087 INFO:teuthology.orchestra.run.smithi057.stdout:103079215177 2024-09-12T03:04:43.087 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:04:43.667 INFO:teuthology.orchestra.run.smithi057.stdout:193273528378 2024-09-12T03:04:43.667 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:04:43.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:43 smithi057 ceph-mon[30361]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:43.717 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:43 smithi057 ceph-mon[30361]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:04:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:43 smithi204 ceph-mon[35013]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:43 smithi204 ceph-mon[35013]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:04:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:43 smithi181 ceph-mon[35320]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:43 smithi181 ceph-mon[35320]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T03:04:44.255 INFO:teuthology.orchestra.run.smithi057.stdout:77309411410 2024-09-12T03:04:44.255 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:04:45.017 INFO:teuthology.orchestra.run.smithi057.stdout:300647710756 2024-09-12T03:04:45.017 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:04:45.104 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.133 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.135 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.142 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.158 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.201 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.235 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.238 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.253 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.260 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.270 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:45.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:45 smithi057 ceph-mon[30361]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:45 smithi204 ceph-mon[35013]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:45 smithi181 ceph-mon[35320]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 -- 172.21.15.57:0/1570055810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b4102360 msgr2=0x7f46b410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1570055810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b4102360 0x7f46b410e890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009cd0 tx=0x7f46ac02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 -- 172.21.15.57:0/1570055810 shutdown_connections 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1570055810 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b4102360 0x7f46b410e890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1570055810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41019c0 0x7f46b4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1570055810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b41079c0 0x7f46b4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 -- 172.21.15.57:0/1570055810 >> 172.21.15.57:0/1570055810 conn(0x7f46b40fd6e0 msgr2=0x7f46b40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 -- 172.21.15.57:0/1570055810 shutdown_connections 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.076+0000 7f46b3e00640 1 -- 172.21.15.57:0/1570055810 wait complete. 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.077+0000 7f46b3e00640 1 Processor -- start 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.077+0000 7f46b3e00640 1 -- start start 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b4102360 0x7f46b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 0x7f46b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4111e10 con 0x7f46b41019c0 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4111c90 con 0x7f46b4102360 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46b4111f90 con 0x7f46b41079c0 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57046/0 (socket says 172.21.15.57:57046) 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 -- 172.21.15.57:0/1959402544 learned_addr learned my addr 172.21.15.57:0/1959402544 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3400640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 0x7f46b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46aa000640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b4102360 0x7f46b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 -- 172.21.15.57:0/1959402544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 msgr2=0x7f46b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 0x7f46b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 -- 172.21.15.57:0/1959402544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b4102360 msgr2=0x7f46b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b4102360 0x7f46b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b2a00640 1 -- 172.21.15.57:0/1959402544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41a6720 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.078+0000 7f46b3400640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 0x7f46b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b2a00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e9f0 tx=0x7f46a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400cd80 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a4010910 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4018990 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41a6a10 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.079+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a6fa0 con 0x7f46b41019c0 2024-09-12T03:04:46.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.080+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b4101e20 con 0x7f46b41019c0 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.082+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f46a4010ac0 con 0x7f46b41019c0 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.083+0000 7f46b8a00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 0x7f46980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.083+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f46a40cf880 con 0x7f46b41019c0 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.084+0000 7f46aa000640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 0x7f46980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.084+0000 7f46aa000640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 0x7f46980a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46b4075f80 tx=0x7f469c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.088+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a409c250 con 0x7f46b41019c0 2024-09-12T03:04:46.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.281+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f46b4075a20 con 0x7f46b41019c0 2024-09-12T03:04:46.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.281+0000 7f46b8a00640 1 -- 172.21.15.57:0/1959402544 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46a409bb50 con 0x7f46b41019c0 2024-09-12T03:04:46.282 INFO:teuthology.orchestra.run.smithi057.stdout:34359738460 2024-09-12T03:04:46.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.283+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 msgr2=0x7f46980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.283+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 0x7f46980a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f46b4075f80 tx=0x7f469c005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:46.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.283+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 msgr2=0x7f46b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.283+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e9f0 tx=0x7f46a400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 shutdown_connections 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f46980a2c90 0x7f46980a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b41079c0 0x7f46b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b4102360 0x7f46b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 --2- 172.21.15.57:0/1959402544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f46b41019c0 0x7f46b40744f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 >> 172.21.15.57:0/1959402544 conn(0x7f46b40fd6e0 msgr2=0x7f46b40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 shutdown_connections 2024-09-12T03:04:46.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.284+0000 7f46b3e00640 1 -- 172.21.15.57:0/1959402544 wait complete. 2024-09-12T03:04:46.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 -- 172.21.15.57:0/3310439675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b701019c0 msgr2=0x7f8b70101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 --2- 172.21.15.57:0/3310439675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b701019c0 0x7f8b70101e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009930 tx=0x7f8b5802a420 comp rx=0 tx=0).stop 2024-09-12T03:04:46.346 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 -- 172.21.15.57:0/3310439675 shutdown_connections 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 --2- 172.21.15.57:0/3310439675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b70102360 0x7f8b7010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 --2- 172.21.15.57:0/3310439675 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b701019c0 0x7f8b70101e20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 --2- 172.21.15.57:0/3310439675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 0x7f8b70107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 -- 172.21.15.57:0/3310439675 >> 172.21.15.57:0/3310439675 conn(0x7f8b700fd740 msgr2=0x7f8b700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 -- 172.21.15.57:0/3310439675 shutdown_connections 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.208+0000 7f8b74c00640 1 -- 172.21.15.57:0/3310439675 wait complete. 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 Processor -- start 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 -- start start 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 0x7f8b70198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 0x7f8b7019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70111e10 con 0x7f8b70102360 2024-09-12T03:04:46.347 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70111c90 con 0x7f8b701079c0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.210+0000 7f8b74c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b70111f90 con 0x7f8b701019c0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 0x7f8b70198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56530/0 (socket says 172.21.15.57:56530) 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 -- 172.21.15.57:0/1935991419 learned_addr learned my addr 172.21.15.57:0/1935991419 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6fe00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 0x7f8b7019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 -- 172.21.15.57:0/1935991419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 msgr2=0x7f8b7019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 0x7f8b7019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 -- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 msgr2=0x7f8b70198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 0x7f8b70198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b6f400640 1 -- 172.21.15.57:0/1935991419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7019d2d0 con 0x7f8b701019c0 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.211+0000 7f8b67e00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 0x7f8b70198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:46.348 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b6f400640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ea60 tx=0x7f8b6000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000cd20 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7019d5c0 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b600035f0 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60010690 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.212+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b701a4ea0 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.213+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b70101e20 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.215+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b600037a0 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.218+0000 7f8b6cc00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 0x7f8b540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.218+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8b600cf950 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.218+0000 7f8b67e00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 0x7f8b540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.219+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b6009c3b0 con 0x7f8b701019c0 2024-09-12T03:04:46.349 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.219+0000 7f8b67e00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 0x7f8b540a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b58031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:46.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.408+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8b70109f90 con 0x7f8b701019c0 2024-09-12T03:04:46.409 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.409+0000 7f8b6cc00640 1 -- 172.21.15.57:0/1935991419 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b6009bcb0 con 0x7f8b701019c0 2024-09-12T03:04:46.410 INFO:teuthology.orchestra.run.smithi057.stdout:158913790018 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 msgr2=0x7f8b540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 0x7f8b540a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b58031510 comp rx=0 tx=0).stop 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 msgr2=0x7f8b70198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ea60 tx=0x7f8b6000ee60 comp rx=0 tx=0).stop 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 shutdown_connections 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8b540a2c10 0x7f8b540a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b701079c0 0x7f8b7019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8b70102360 0x7f8b70198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 --2- 172.21.15.57:0/1935991419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b701019c0 0x7f8b70198320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:46.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.411+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 >> 172.21.15.57:0/1935991419 conn(0x7f8b700fd740 msgr2=0x7f8b700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:46.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.412+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 shutdown_connections 2024-09-12T03:04:46.413 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:46.412+0000 7f8b74c00640 1 -- 172.21.15.57:0/1935991419 wait complete. 2024-09-12T03:04:46.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:46 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1959402544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:04:46.842 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:46 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1935991419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:04:46.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:46 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1959402544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:04:46.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:46 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1935991419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:04:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:46 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1959402544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:04:46.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:46 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1935991419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:04:47.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:47 smithi057 ceph-mon[30361]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:04:47.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:47 smithi204 ceph-mon[35013]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:04:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:47 smithi181 ceph-mon[35320]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.027+0000 7fb767200640 1 -- 172.21.15.57:0/2613538788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 msgr2=0x7fb768101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.027+0000 7fb767200640 1 --2- 172.21.15.57:0/2613538788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb768101e20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb754009930 tx=0x7fb75402a420 comp rx=0 tx=0).stop 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 -- 172.21.15.57:0/2613538788 shutdown_connections 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 --2- 172.21.15.57:0/2613538788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 --2- 172.21.15.57:0/2613538788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb768101e20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 --2- 172.21.15.57:0/2613538788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 0x7fb768107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 -- 172.21.15.57:0/2613538788 >> 172.21.15.57:0/2613538788 conn(0x7fb7680fd740 msgr2=0x7fb7680ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:48.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 -- 172.21.15.57:0/2613538788 shutdown_connections 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.028+0000 7fb767200640 1 -- 172.21.15.57:0/2613538788 wait complete. 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 Processor -- start 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 -- start start 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb76819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 0x7fb76819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb768075320 con 0x7fb768102360 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680751a0 con 0x7fb7681079c0 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb767200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7680754a0 con 0x7fb7681019c0 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb765400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb765400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42784/0 (socket says 172.21.15.57:42784) 2024-09-12T03:04:48.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb765400640 1 -- 172.21.15.57:0/3670351659 learned_addr learned my addr 172.21.15.57:0/3670351659 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb765e00640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb76819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.029+0000 7fb766800640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 0x7fb76819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 -- 172.21.15.57:0/3670351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 msgr2=0x7fb76819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb76819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 -- 172.21.15.57:0/3670351659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 msgr2=0x7fb76819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 0x7fb76819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 -- 172.21.15.57:0/3670351659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76819f540 con 0x7fb768102360 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765e00640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb76819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.030+0000 7fb765400640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb754009230 tx=0x7fb754004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.031+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75403f030 con 0x7fb768102360 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.031+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb754004440 con 0x7fb768102360 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.031+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76819f7d0 con 0x7fb768102360 2024-09-12T03:04:48.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.031+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754035070 con 0x7fb768102360 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.031+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a7110 con 0x7fb768102360 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.032+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768101e20 con 0x7fb768102360 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.034+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb754042030 con 0x7fb768102360 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.035+0000 7fb75ea00640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 0x7fb74c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.035+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb7540ef920 con 0x7fb768102360 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.035+0000 7fb765e00640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 0x7fb74c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.036+0000 7fb765e00640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 0x7fb74c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb758009f40 tx=0x7fb758005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:48.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.039+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7540bc2f0 con 0x7fb768102360 2024-09-12T03:04:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.225+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb7681a7310 con 0x7fb768102360 2024-09-12T03:04:48.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.226+0000 7fb75ea00640 1 -- 172.21.15.57:0/3670351659 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb7540bbbf0 con 0x7fb768102360 2024-09-12T03:04:48.226 INFO:teuthology.orchestra.run.smithi057.stdout:244813135922 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 msgr2=0x7fb74c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 0x7fb74c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb758009f40 tx=0x7fb758005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 msgr2=0x7fb76819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb754009230 tx=0x7fb754004040 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 shutdown_connections 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb74c0a2c10 0x7fb74c0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7681079c0 0x7fb76819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb768102360 0x7fb76819aad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 --2- 172.21.15.57:0/3670351659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7681019c0 0x7fb76819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 >> 172.21.15.57:0/3670351659 conn(0x7fb7680fd740 msgr2=0x7fb7680ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 shutdown_connections 2024-09-12T03:04:48.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.228+0000 7fb767200640 1 -- 172.21.15.57:0/3670351659 wait complete. 2024-09-12T03:04:48.821 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:48 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3670351659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 -- 172.21.15.57:0/3366537049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d01049a0 msgr2=0x7f20d01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 --2- 172.21.15.57:0/3366537049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d01049a0 0x7f20d01089f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009900 tx=0x7f20c002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 -- 172.21.15.57:0/3366537049 shutdown_connections 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 --2- 172.21.15.57:0/3366537049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0109120 0x7f20d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 --2- 172.21.15.57:0/3366537049 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d01049a0 0x7f20d01089f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 --2- 172.21.15.57:0/3366537049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d0103ff0 0x7f20d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 -- 172.21.15.57:0/3366537049 >> 172.21.15.57:0/3366537049 conn(0x7f20d00ffa90 msgr2=0x7f20d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 -- 172.21.15.57:0/3366537049 shutdown_connections 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.821+0000 7f20d8800640 1 -- 172.21.15.57:0/3366537049 wait complete. 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 Processor -- start 2024-09-12T03:04:48.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 -- start start 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 0x7f20d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d01049a0 0x7f20d019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0114140 con 0x7f20d0109120 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f20d0113fc0 con 0x7f20d01049a0 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.822+0000 7f20d8800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f20d01142c0 con 0x7f20d0103ff0 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d01049a0 0x7f20d019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42830/0 (socket says 172.21.15.57:42830) 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 0x7f20d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d6800640 1 -- 172.21.15.57:0/2844010096 learned_addr learned my addr 172.21.15.57:0/2844010096 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 -- 172.21.15.57:0/2844010096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d01049a0 msgr2=0x7f20d019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d01049a0 0x7f20d019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 -- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 msgr2=0x7f20d019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d5e00640 1 -- 172.21.15.57:0/2844010096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d019f5d0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.823+0000 7f20d6800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20d5e00640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 0x7f20d019a620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e9f0 tx=0x7f20c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c400cd80 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d019f8c0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4010670 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.824+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d01a71a0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.825+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d01043d0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.827+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f20c40037a0 con 0x7f20d0103ff0 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.828+0000 7f20caa00640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 0x7f20b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:48.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.828+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f20c40cf7f0 con 0x7f20d0103ff0 2024-09-12T03:04:48.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.829+0000 7f20d5400640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 0x7f20b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:48.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.829+0000 7f20d5400640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 0x7f20b80a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009230 tx=0x7f20c00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:48.850 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.830+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20c409c1c0 con 0x7f20d0103ff0 2024-09-12T03:04:48.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:48 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3670351659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:04:48.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:48 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3670351659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:04:49.023 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.023+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f20d019b980 con 0x7f20d0103ff0 2024-09-12T03:04:49.024 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.024+0000 7f20caa00640 1 -- 172.21.15.57:0/2844010096 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20d019b980 con 0x7f20d0103ff0 2024-09-12T03:04:49.024 INFO:teuthology.orchestra.run.smithi057.stdout:55834574936 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 msgr2=0x7f20b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 0x7f20b80a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009230 tx=0x7f20c00058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 msgr2=0x7f20d019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 0x7f20d019a620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20c400e9f0 tx=0x7f20c400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 shutdown_connections 2024-09-12T03:04:49.027 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f20b80a2c90 0x7f20b80a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f20d0109120 0x7f20d019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20d01049a0 0x7f20d019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 --2- 172.21.15.57:0/2844010096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20d0103ff0 0x7f20d019a620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.026+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 >> 172.21.15.57:0/2844010096 conn(0x7f20d00ffa90 msgr2=0x7f20d0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.027+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 shutdown_connections 2024-09-12T03:04:49.028 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.027+0000 7f20d8800640 1 -- 172.21.15.57:0/2844010096 wait complete. 2024-09-12T03:04:49.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:49 smithi204 ceph-mon[35013]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:04:49.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:49 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3385856703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:04:49.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:49 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2844010096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:04:49.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:49 smithi057 ceph-mon[30361]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:04:49.807 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:49 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3385856703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:04:49.808 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:49 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2844010096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:04:49.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.923+0000 7f696a800640 1 -- 172.21.15.57:0/876051171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 msgr2=0x7f6964114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.923+0000 7f696a800640 1 --2- 172.21.15.57:0/876051171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f6964114e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f695c009ca0 tx=0x7f695c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 -- 172.21.15.57:0/876051171 shutdown_connections 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 --2- 172.21.15.57:0/876051171 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f6964114e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 --2- 172.21.15.57:0/876051171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 0x7f69641124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 --2- 172.21.15.57:0/876051171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f6964076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 -- 172.21.15.57:0/876051171 >> 172.21.15.57:0/876051171 conn(0x7f69640ffa90 msgr2=0x7f6964101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 -- 172.21.15.57:0/876051171 shutdown_connections 2024-09-12T03:04:49.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.924+0000 7f696a800640 1 -- 172.21.15.57:0/876051171 wait complete. 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 Processor -- start 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 -- start start 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 0x7f69641a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f696419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f69641185d0 con 0x7f6964112a30 2024-09-12T03:04:49.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6964118450 con 0x7f6964077040 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.925+0000 7f696a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6964118750 con 0x7f6964076720 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:37460/0 (socket says 172.21.15.57:37460) 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 -- 172.21.15.57:0/23907100 learned_addr learned my addr 172.21.15.57:0/23907100 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963e00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f696419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:49.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f695be00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 0x7f69641a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 -- 172.21.15.57:0/23907100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 msgr2=0x7f69641a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 0x7f69641a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 -- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 msgr2=0x7f696419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f696419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963400640 1 -- 172.21.15.57:0/23907100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696419efa0 con 0x7f6964076720 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.926+0000 7f6963e00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f696419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f6963400640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9f0 tx=0x7f695400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:49.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695400cd80 con 0x7f6964076720 2024-09-12T03:04:49.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696419f290 con 0x7f6964076720 2024-09-12T03:04:49.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6954005500 con 0x7f6964076720 2024-09-12T03:04:49.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954010690 con 0x7f6964076720 2024-09-12T03:04:49.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.927+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641ab580 con 0x7f6964076720 2024-09-12T03:04:49.929 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.928+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6964076b00 con 0x7f6964076720 2024-09-12T03:04:49.930 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.930+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f69540046c0 con 0x7f6964076720 2024-09-12T03:04:49.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.932+0000 7f6960c00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 0x7f69480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:49.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.932+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f69540cf990 con 0x7f6964076720 2024-09-12T03:04:49.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.932+0000 7f695be00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 0x7f69480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:49.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.932+0000 7f695be00640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 0x7f69480a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69640792b0 tx=0x7f694c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:49.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:49.933+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f695409c360 con 0x7f6964076720 2024-09-12T03:04:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:49 smithi181 ceph-mon[35320]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-12T03:04:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:49 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3385856703' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:04:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:49 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2844010096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:04:50.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.123+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f696419fda0 con 0x7f6964076720 2024-09-12T03:04:50.125 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.124+0000 7f6960c00640 1 -- 172.21.15.57:0/23907100 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f695409bc60 con 0x7f6964076720 2024-09-12T03:04:50.125 INFO:teuthology.orchestra.run.smithi057.stdout:214748364854 2024-09-12T03:04:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 msgr2=0x7f69480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 0x7f69480a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69640792b0 tx=0x7f694c005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:50.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 msgr2=0x7f69641a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f695400e9f0 tx=0x7f695400edf0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 shutdown_connections 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f69480a2c90 0x7f69480a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6964112a30 0x7f696419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6964077040 0x7f69641a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 --2- 172.21.15.57:0/23907100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6964076720 0x7f69641a46b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 >> 172.21.15.57:0/23907100 conn(0x7f69640ffa90 msgr2=0x7f6964101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 shutdown_connections 2024-09-12T03:04:50.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.127+0000 7f696a800640 1 -- 172.21.15.57:0/23907100 wait complete. 2024-09-12T03:04:50.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790016 got 158913790018 for osd.5 2024-09-12T03:04:50.177 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:50.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:50 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/23907100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:04:50.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:50 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/23907100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:04:50.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:50 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/23907100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:04:51.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574936 for osd.1 2024-09-12T03:04:51.038 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:51.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 -- 172.21.15.57:0/1747983981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c109120 msgr2=0x7fb47c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 --2- 172.21.15.57:0/1747983981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c109120 0x7fb47c110ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb474009900 tx=0x7fb47402a410 comp rx=0 tx=0).stop 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 -- 172.21.15.57:0/1747983981 shutdown_connections 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 --2- 172.21.15.57:0/1747983981 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c109120 0x7fb47c110ca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 --2- 172.21.15.57:0/1747983981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c1049a0 0x7fb47c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 --2- 172.21.15.57:0/1747983981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 0x7fb47c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.382+0000 7fb480a00640 1 -- 172.21.15.57:0/1747983981 >> 172.21.15.57:0/1747983981 conn(0x7fb47c0ffa90 msgr2=0x7fb47c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.383+0000 7fb480a00640 1 -- 172.21.15.57:0/1747983981 shutdown_connections 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.383+0000 7fb480a00640 1 -- 172.21.15.57:0/1747983981 wait complete. 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.383+0000 7fb480a00640 1 Processor -- start 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 -- start start 2024-09-12T03:04:51.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 0x7fb47c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 0x7fb47c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c114150 con 0x7fb47c1049a0 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c113fd0 con 0x7fb47c109120 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb480a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1142d0 con 0x7fb47c103ff0 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb471400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb471400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42862/0 (socket says 172.21.15.57:42862) 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb471400640 1 -- 172.21.15.57:0/580729667 learned_addr learned my addr 172.21.15.57:0/580729667 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb47a800640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 0x7fb47c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.384+0000 7fb479e00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 0x7fb47c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 -- 172.21.15.57:0/580729667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 msgr2=0x7fb47c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 0x7fb47c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 -- 172.21.15.57:0/580729667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 msgr2=0x7fb47c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 0x7fb47c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 -- 172.21.15.57:0/580729667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c19f5c0 con 0x7fb47c1049a0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb47a800640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 0x7fb47c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb471400640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d950 tx=0x7fb46400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464017030 con 0x7fb47c1049a0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c19f8b0 con 0x7fb47c1049a0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb464004020 con 0x7fb47c1049a0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.385+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4640047a0 con 0x7fb47c1049a0 2024-09-12T03:04:51.118 INFO:teuthology.orchestra.run.smithi057.stdout:103079215181 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.386+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a7190 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.387+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c1043d0 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.388+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb4640041d0 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.390+0000 7fb473400640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 0x7fb4600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.390+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fb4640cee90 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.390+0000 7fb479e00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 0x7fb4600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.391+0000 7fb479e00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 0x7fb4600a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.393+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb46409b860 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.582+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb47c02d7c0 con 0x7fb47c1049a0 2024-09-12T03:04:51.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.582+0000 7fb473400640 1 -- 172.21.15.57:0/580729667 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb46409b160 con 0x7fb47c1049a0 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 msgr2=0x7fb4600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 0x7fb4600a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb46c005fd0 tx=0x7fb46c009500 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 msgr2=0x7fb47c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d950 tx=0x7fb46400de10 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 shutdown_connections 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fb4600a2c10 0x7fb4600a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47c109120 0x7fb47c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb47c1049a0 0x7fb47c19ab50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 --2- 172.21.15.57:0/580729667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb47c103ff0 0x7fb47c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 >> 172.21.15.57:0/580729667 conn(0x7fb47c0ffa90 msgr2=0x7fb47c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 shutdown_connections 2024-09-12T03:04:51.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:50.585+0000 7fb480a00640 1 -- 172.21.15.57:0/580729667 wait complete. 2024-09-12T03:04:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 -- 172.21.15.57:0/378076997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 msgr2=0x7f9400108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.594 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 --2- 172.21.15.57:0/378076997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f9400108830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 -- 172.21.15.57:0/378076997 shutdown_connections 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 --2- 172.21.15.57:0/378076997 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 0x7f9400110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 --2- 172.21.15.57:0/378076997 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f9400108830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 --2- 172.21.15.57:0/378076997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f9400104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 -- 172.21.15.57:0/378076997 >> 172.21.15.57:0/378076997 conn(0x7f94000ff8d0 msgr2=0x7f9400101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 -- 172.21.15.57:0/378076997 shutdown_connections 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.349+0000 7f9406600640 1 -- 172.21.15.57:0/378076997 wait complete. 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 Processor -- start 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 -- start start 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f940019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 0x7f940019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113fc0 con 0x7f94001047e0 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9400113e40 con 0x7f9400108f60 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.350+0000 7f9406600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9400114140 con 0x7f9400103e30 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f93ffe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f940019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:37416/0 (socket says 172.21.15.57:37416) 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 -- 172.21.15.57:0/3385856703 learned_addr learned my addr 172.21.15.57:0/3385856703 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405c00640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 0x7f940019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 -- 172.21.15.57:0/3385856703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 msgr2=0x7f940019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 0x7f940019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 -- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 msgr2=0x7f940019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f940019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 -- 172.21.15.57:0/3385856703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940019f450 con 0x7f9400103e30 2024-09-12T03:04:51.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f93ffe00640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f940019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.351+0000 7f9405200640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93f400e9f0 tx=0x7f93f400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.352+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f400cd80 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.352+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940019f740 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.352+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f4005500 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.352+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4010690 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.352+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a7020 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.353+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9400104210 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.355+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f93f40046c0 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.356+0000 7f93fd600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 0x7f93e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.356+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f93f40cfc10 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.357+0000 7f93ffe00640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 0x7f93e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.357+0000 7f93ffe00640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 0x7f93e80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009230 tx=0x7f93ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.358+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93f40541b0 con 0x7f9400103e30 2024-09-12T03:04:51.597 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.550+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f940010c1a0 con 0x7f9400103e30 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stdout:330712481827 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.550+0000 7f93fd600640 1 -- 172.21.15.57:0/3385856703 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f93f409c5e0 con 0x7f9400103e30 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 msgr2=0x7f93e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 0x7f93e80a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009230 tx=0x7f93ec0058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 msgr2=0x7f940019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93f400e9f0 tx=0x7f93f400edf0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 shutdown_connections 2024-09-12T03:04:51.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f93e80a2c10 0x7f93e80a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9400108f60 0x7f940019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94001047e0 0x7f940019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 --2- 172.21.15.57:0/3385856703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9400103e30 0x7f940019a4a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 >> 172.21.15.57:0/3385856703 conn(0x7f94000ff8d0 msgr2=0x7f9400100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.552+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 shutdown_connections 2024-09-12T03:04:51.599 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:48.553+0000 7f9406600640 1 -- 172.21.15.57:0/3385856703 wait complete. 2024-09-12T03:04:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:51 smithi057 ceph-mon[30361]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T03:04:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:51 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/580729667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:04:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:51 smithi204 ceph-mon[35013]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T03:04:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:51 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/580729667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:04:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:51 smithi181 ceph-mon[35320]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T03:04:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:51 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/580729667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:04:52.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481823 got 330712481827 for osd.11 2024-09-12T03:04:52.480 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 -- 172.21.15.57:0/3160639074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 msgr2=0x7fa00c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 --2- 172.21.15.57:0/3160639074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c1043d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009900 tx=0x7f9ffc02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 -- 172.21.15.57:0/3160639074 shutdown_connections 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 --2- 172.21.15.57:0/3160639074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 0x7fa00c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 --2- 172.21.15.57:0/3160639074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 --2- 172.21.15.57:0/3160639074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c1043d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 -- 172.21.15.57:0/3160639074 >> 172.21.15.57:0/3160639074 conn(0x7fa00c0ffa90 msgr2=0x7fa00c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 -- 172.21.15.57:0/3160639074 shutdown_connections 2024-09-12T03:04:52.683 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.682+0000 7fa013200640 1 -- 172.21.15.57:0/3160639074 wait complete. 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.683+0000 7fa013200640 1 Processor -- start 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.683+0000 7fa013200640 1 -- start start 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.683+0000 7fa013200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.683+0000 7fa013200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa013200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 0x7fa00c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa013200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c114140 con 0x7fa00c103ff0 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa013200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c113fc0 con 0x7fa00c109120 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa013200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1142c0 con 0x7fa00c1049a0 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.685 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42876/0 (socket says 172.21.15.57:42876) 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00be00640 1 -- 172.21.15.57:0/1112185298 learned_addr learned my addr 172.21.15.57:0/1112185298 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa011200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 0x7fa00c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 -- 172.21.15.57:0/1112185298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 msgr2=0x7fa00c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 0x7fa00c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 -- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 msgr2=0x7fa00c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.684+0000 7fa00b400640 1 -- 172.21.15.57:0/1112185298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c19f670 con 0x7fa00c1049a0 2024-09-12T03:04:52.686 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa00b400640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c19ac00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800e9f0 tx=0x7f9ff800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800ccc0 con 0x7fa00c1049a0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ff8005020 con 0x7fa00c1049a0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80049e0 con 0x7fa00c1049a0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c19f960 con 0x7fa00c1049a0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.685+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c1a7240 con 0x7fa00c1049a0 2024-09-12T03:04:52.687 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.686+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00c1043d0 con 0x7fa00c1049a0 2024-09-12T03:04:52.689 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.689+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9ff8005560 con 0x7fa00c1049a0 2024-09-12T03:04:52.690 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.690+0000 7fa008c00640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 0x7f9ff00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.690+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f9ff80cf580 con 0x7fa00c1049a0 2024-09-12T03:04:52.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.690+0000 7fa00be00640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 0x7f9ff00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.691 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.691+0000 7fa00be00640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 0x7f9ff00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0096c0 tx=0x7f9ffc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.694+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ff80051d0 con 0x7fa00c1049a0 2024-09-12T03:04:52.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.883+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa00c10bd70 con 0x7fa00c1049a0 2024-09-12T03:04:52.884 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.883+0000 7fa008c00640 1 -- 172.21.15.57:0/1112185298 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9ff809bf50 con 0x7fa00c1049a0 2024-09-12T03:04:52.884 INFO:teuthology.orchestra.run.smithi057.stdout:193273528381 2024-09-12T03:04:52.886 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 msgr2=0x7f9ff00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 0x7f9ff00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0096c0 tx=0x7f9ffc005a20 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 msgr2=0x7fa00c19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c19ac00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800e9f0 tx=0x7f9ff800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 shutdown_connections 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f9ff00a2c10 0x7f9ff00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa00c109120 0x7fa00c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa00c1049a0 0x7fa00c19ac00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 --2- 172.21.15.57:0/1112185298 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa00c103ff0 0x7fa00c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 >> 172.21.15.57:0/1112185298 conn(0x7fa00c0ffa90 msgr2=0x7fa00c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 shutdown_connections 2024-09-12T03:04:52.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:52.886+0000 7fa013200640 1 -- 172.21.15.57:0/1112185298 wait complete. 2024-09-12T03:04:53.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:53 smithi057 ceph-mon[30361]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:53.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:53 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1112185298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:04:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:53 smithi204 ceph-mon[35013]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:53 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1112185298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:04:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:53 smithi181 ceph-mon[35320]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-12T03:04:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:53 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1112185298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:04:54.042 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528378 got 193273528381 for osd.6 2024-09-12T03:04:54.042 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.106+0000 7fad23e00640 1 -- 172.21.15.57:0/2916953040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 msgr2=0x7fad1c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.106+0000 7fad23e00640 1 --2- 172.21.15.57:0/2916953040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c1043d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fad10009900 tx=0x7fad1002a410 comp rx=0 tx=0).stop 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 -- 172.21.15.57:0/2916953040 shutdown_connections 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 --2- 172.21.15.57:0/2916953040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c109120 0x7fad1c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 --2- 172.21.15.57:0/2916953040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c1049a0 0x7fad1c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 --2- 172.21.15.57:0/2916953040 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c1043d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 -- 172.21.15.57:0/2916953040 >> 172.21.15.57:0/2916953040 conn(0x7fad1c0ffa90 msgr2=0x7fad1c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 -- 172.21.15.57:0/2916953040 shutdown_connections 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.107+0000 7fad23e00640 1 -- 172.21.15.57:0/2916953040 wait complete. 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 Processor -- start 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 -- start start 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1049a0 0x7fad1c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c109120 0x7fad1c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c114150 con 0x7fad1c103ff0 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c113fd0 con 0x7fad1c109120 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad23e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1142d0 con 0x7fad1c1049a0 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad21400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.112 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad21400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42896/0 (socket says 172.21.15.57:42896) 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.108+0000 7fad21400640 1 -- 172.21.15.57:0/1180509950 learned_addr learned my addr 172.21.15.57:0/1180509950 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c109120 0x7fad1c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad20a00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1049a0 0x7fad1c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 -- 172.21.15.57:0/1180509950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1049a0 msgr2=0x7fad1c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1049a0 0x7fad1c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 -- 172.21.15.57:0/1180509950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c109120 msgr2=0x7fad1c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c109120 0x7fad1c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 -- 172.21.15.57:0/1180509950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c19f5e0 con 0x7fad1c103ff0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad21400640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad10006010 tx=0x7fad100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.109+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1003f030 con 0x7fad1c103ff0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.110+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10003ce0 con 0x7fad1c103ff0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.110+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c19f870 con 0x7fad1c103ff0 2024-09-12T03:04:55.113 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.110+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10035070 con 0x7fad1c103ff0 2024-09-12T03:04:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.110+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c1a71b0 con 0x7fad1c103ff0 2024-09-12T03:04:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.111+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1c1054d0 con 0x7fad1c103ff0 2024-09-12T03:04:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.113+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad10042030 con 0x7fad1c103ff0 2024-09-12T03:04:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.114+0000 7fad1a000640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 0x7fad040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.114+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fad100efab0 con 0x7fad1c103ff0 2024-09-12T03:04:55.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.114+0000 7fad20a00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 0x7fad040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.115+0000 7fad20a00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 0x7fad040a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0ffeb0 tx=0x7fad08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.119+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad100bbef0 con 0x7fad1c103ff0 2024-09-12T03:04:55.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.308+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fad1c10c300 con 0x7fad1c103ff0 2024-09-12T03:04:55.309 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.309+0000 7fad1a000640 1 -- 172.21.15.57:0/1180509950 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fad100c01f0 con 0x7fad1c103ff0 2024-09-12T03:04:55.310 INFO:teuthology.orchestra.run.smithi057.stdout:137438953545 2024-09-12T03:04:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.311+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 msgr2=0x7fad040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 0x7fad040a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0ffeb0 tx=0x7fad08005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:55.312 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 msgr2=0x7fad1c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad10006010 tx=0x7fad100042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 shutdown_connections 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fad040a2c10 0x7fad040a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad1c109120 0x7fad1c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1049a0 0x7fad1c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 --2- 172.21.15.57:0/1180509950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad1c103ff0 0x7fad1c19a630 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 >> 172.21.15.57:0/1180509950 conn(0x7fad1c0ffa90 msgr2=0x7fad1c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 shutdown_connections 2024-09-12T03:04:55.313 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.312+0000 7fad23e00640 1 -- 172.21.15.57:0/1180509950 wait complete. 2024-09-12T03:04:55.387 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 -- 172.21.15.57:0/1396575345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee01019d0 msgr2=0x7fdee0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 --2- 172.21.15.57:0/1396575345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee01019d0 0x7fdee0101e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009890 tx=0x7fdecc02a360 comp rx=0 tx=0).stop 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 -- 172.21.15.57:0/1396575345 shutdown_connections 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 --2- 172.21.15.57:0/1396575345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee0102370 0x7fdee010e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 --2- 172.21.15.57:0/1396575345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee01019d0 0x7fdee0101e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 --2- 172.21.15.57:0/1396575345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01079d0 0x7fdee0107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 -- 172.21.15.57:0/1396575345 >> 172.21.15.57:0/1396575345 conn(0x7fdee00fd710 msgr2=0x7fdee00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.388 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 -- 172.21.15.57:0/1396575345 shutdown_connections 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.348+0000 7fdee6000640 1 -- 172.21.15.57:0/1396575345 wait complete. 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.349+0000 7fdee6000640 1 Processor -- start 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.349+0000 7fdee6000640 1 -- start start 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01019d0 0x7fdee019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee0102370 0x7fdee019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0111da0 con 0x7fdee01019d0 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0111c20 con 0x7fdee01079d0 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee0111f20 con 0x7fdee0102370 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.389 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41706/0 (socket says 172.21.15.57:41706) 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 -- 172.21.15.57:0/465372906 learned_addr learned my addr 172.21.15.57:0/465372906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee4c00640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01019d0 0x7fdee019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdedfe00640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee0102370 0x7fdee019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 -- 172.21.15.57:0/465372906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee0102370 msgr2=0x7fdee019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee0102370 0x7fdee019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 -- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01019d0 msgr2=0x7fdee019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.350+0000 7fdee5600640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01019d0 0x7fdee019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdee5600640 1 -- 172.21.15.57:0/465372906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee019f4d0 con 0x7fdee01079d0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdee5600640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fded800eb10 tx=0x7fded800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8004b20 con 0x7fdee01079d0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fded8004cc0 con 0x7fdee01079d0 2024-09-12T03:04:55.390 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee019f7c0 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8010760 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.351+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee01a70a0 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.352+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee0101e30 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.356+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fded8044b60 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.359+0000 7fdedd600640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 0x7fdec80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.359+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fded80cfce0 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.360+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fded80d0160 con 0x7fdee01079d0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.360+0000 7fdee4c00640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 0x7fdec80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.391 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.361+0000 7fdee4c00640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 0x7fdec80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fded40096c0 tx=0x7fded4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.551 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.550+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdee019b880 con 0x7fdee01079d0 2024-09-12T03:04:55.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.551+0000 7fdedd600640 1 -- 172.21.15.57:0/465372906 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdee019b880 con 0x7fdee01079d0 2024-09-12T03:04:55.552 INFO:teuthology.orchestra.run.smithi057.stdout:77309411413 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 msgr2=0x7fdec80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 0x7fdec80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fded40096c0 tx=0x7fded4009260 comp rx=0 tx=0).stop 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 msgr2=0x7fdee019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fded800eb10 tx=0x7fded800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 shutdown_connections 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fdec80a2c10 0x7fdec80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.554 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdee01079d0 0x7fdee019edf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee0102370 0x7fdee019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 --2- 172.21.15.57:0/465372906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdee01019d0 0x7fdee019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.553+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 >> 172.21.15.57:0/465372906 conn(0x7fdee00fd710 msgr2=0x7fdee00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.554+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 shutdown_connections 2024-09-12T03:04:55.555 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:55.554+0000 7fdee6000640 1 -- 172.21.15.57:0/465372906 wait complete. 2024-09-12T03:04:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:55 smithi057 ceph-mon[30361]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:04:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:55 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1180509950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:04:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:55 smithi204 ceph-mon[35013]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:04:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:55 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1180509950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:04:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:55 smithi181 ceph-mon[35320]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T03:04:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:55 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1180509950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:04:56.416 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135920 got 244813135922 for osd.8 2024-09-12T03:04:56.417 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:56.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 -- 172.21.15.57:0/154244544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8112a30 msgr2=0x7f65e8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 --2- 172.21.15.57:0/154244544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8112a30 0x7f65e8114e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009ca0 tx=0x7f65e002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 -- 172.21.15.57:0/154244544 shutdown_connections 2024-09-12T03:04:56.633 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 --2- 172.21.15.57:0/154244544 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8112a30 0x7f65e8114e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 --2- 172.21.15.57:0/154244544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8077040 0x7f65e81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 --2- 172.21.15.57:0/154244544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8076720 0x7f65e8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 -- 172.21.15.57:0/154244544 >> 172.21.15.57:0/154244544 conn(0x7f65e80ffa90 msgr2=0x7f65e8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 -- 172.21.15.57:0/154244544 shutdown_connections 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.611+0000 7f65eea00640 1 -- 172.21.15.57:0/154244544 wait complete. 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 Processor -- start 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 -- start start 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8076720 0x7f65e81a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8077040 0x7f65e81a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f65e81185d0 con 0x7f65e8076720 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8118450 con 0x7f65e8112a30 2024-09-12T03:04:56.634 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.612+0000 7f65eea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f65e8118750 con 0x7f65e8077040 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65e7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8076720 0x7f65e81a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8077040 0x7f65e81a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41726/0 (socket says 172.21.15.57:41726) 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 -- 172.21.15.57:0/3934864466 learned_addr learned my addr 172.21.15.57:0/3934864466 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 -- 172.21.15.57:0/3934864466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8077040 msgr2=0x7f65e81a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8077040 0x7f65e81a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 -- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8076720 msgr2=0x7f65e81a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8076720 0x7f65e81a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.613+0000 7f65eca00640 1 -- 172.21.15.57:0/3934864466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e819efa0 con 0x7f65e8112a30 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65eca00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005e30 tx=0x7f65e0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0031650 con 0x7f65e8112a30 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e00317f0 con 0x7f65e8112a30 2024-09-12T03:04:56.635 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e819f230 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e002fa10 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.614+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e81ab580 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.615+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e8076b00 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.617+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65e002fbb0 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.618+0000 7f65e4c00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.619+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f65e00f0930 con 0x7f65e8112a30 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.619+0000 7f65e7e00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.619+0000 7f65e7e00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f65e819f9b0 tx=0x7f65d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:56.636 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.620+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65e00bd410 con 0x7f65e8112a30 2024-09-12T03:04:56.637 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:56 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/465372906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:04:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.811+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f65e819fda0 con 0x7f65e8112a30 2024-09-12T03:04:56.812 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.812+0000 7f65e4c00640 1 -- 172.21.15.57:0/3934864466 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f65e00319a0 con 0x7f65e8112a30 2024-09-12T03:04:56.813 INFO:teuthology.orchestra.run.smithi057.stdout:279172874285 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 msgr2=0x7f65cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f65e819f9b0 tx=0x7f65d8005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 msgr2=0x7f65e819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005e30 tx=0x7f65e0003f10 comp rx=0 tx=0).stop 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 shutdown_connections 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f65cc0a2c10 0x7f65cc0a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e8112a30 0x7f65e819e870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e8077040 0x7f65e81a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 --2- 172.21.15.57:0/3934864466 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f65e8076720 0x7f65e81a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.814+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 >> 172.21.15.57:0/3934864466 conn(0x7f65e80ffa90 msgr2=0x7f65e8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.815+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 shutdown_connections 2024-09-12T03:04:56.816 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:04:56.815+0000 7f65eea00640 1 -- 172.21.15.57:0/3934864466 wait complete. 2024-09-12T03:04:56.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:56 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/465372906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:04:56.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:56 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/465372906' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:04:57.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738458 got 34359738460 for osd.0 2024-09-12T03:04:57.672 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:57 smithi057 ceph-mon[30361]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:57 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3934864466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:04:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:57 smithi204 ceph-mon[35013]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:57.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:57 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3934864466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:04:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:57 smithi181 ceph-mon[35320]: pgmap v364: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:04:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:57 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3934864466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:04:58.031 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953540 got 137438953545 for osd.4 2024-09-12T03:04:58.031 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:58.362 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411410 got 77309411413 for osd.2 2024-09-12T03:04:58.362 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:58.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364853 got 214748364854 for osd.7 2024-09-12T03:04:58.760 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:59.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215177 got 103079215181 for osd.3 2024-09-12T03:04:59.105 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:59.591 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:59.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874281 got 279172874285 for osd.9 2024-09-12T03:04:59.623 DEBUG:teuthology.parallel:result is None 2024-09-12T03:04:59.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:04:59 smithi057 ceph-mon[30361]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:04:59.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:04:59 smithi204 ceph-mon[35013]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:04:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:04:59 smithi181 ceph-mon[35320]: pgmap v365: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 -- 172.21.15.57:0/2966821723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b01034b0 msgr2=0x7fa8b010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/2966821723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b01034b0 0x7fa8b010b9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa898009900 tx=0x7fa89802a480 comp rx=0 tx=0).stop 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 -- 172.21.15.57:0/2966821723 shutdown_connections 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/2966821723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b010bfc0 0x7fa8b010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/2966821723 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b01034b0 0x7fa8b010b9a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/2966821723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 -- 172.21.15.57:0/2966821723 >> 172.21.15.57:0/2966821723 conn(0x7fa8b00fc610 msgr2=0x7fa8b00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:00.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 -- 172.21.15.57:0/2966821723 shutdown_connections 2024-09-12T03:05:00.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.175+0000 7fa8b4e00640 1 -- 172.21.15.57:0/2966821723 wait complete. 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.176+0000 7fa8b4e00640 1 Processor -- start 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.176+0000 7fa8b4e00640 1 -- start start 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b01034b0 0x7fa8b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 0x7fa8b019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0111e10 con 0x7fa8b010bfc0 2024-09-12T03:05:00.178 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0111c90 con 0x7fa8b01034b0 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8b4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0111f90 con 0x7fa8b0102b90 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 0x7fa8b019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8af400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8af400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:39710/0 (socket says 172.21.15.57:39710) 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8af400640 1 -- 172.21.15.57:0/3416603710 learned_addr learned my addr 172.21.15.57:0/3416603710 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8a7e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b01034b0 0x7fa8b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 -- 172.21.15.57:0/3416603710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 msgr2=0x7fa8b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 -- 172.21.15.57:0/3416603710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b01034b0 msgr2=0x7fa8b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b01034b0 0x7fa8b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8afe00640 1 -- 172.21.15.57:0/3416603710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b019d260 con 0x7fa8b010bfc0 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.177+0000 7fa8af400640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:00.179 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8afe00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 0x7fa8b019cb80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800da10 tx=0x7fa8a800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a800bd70 con 0x7fa8b010bfc0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a8003e50 con 0x7fa8b010bfc0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b019d550 con 0x7fa8b010bfc0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80106a0 con 0x7fa8b010bfc0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.178+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01a4e30 con 0x7fa8b010bfc0 2024-09-12T03:05:00.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.179+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b0102f70 con 0x7fa8b010bfc0 2024-09-12T03:05:00.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.181+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa8a8003970 con 0x7fa8b010bfc0 2024-09-12T03:05:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.182+0000 7fa8acc00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 0x7fa8940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.183+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fa8a80cf880 con 0x7fa8b010bfc0 2024-09-12T03:05:00.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.183+0000 7fa8af400640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 0x7fa8940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.183+0000 7fa8af400640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 0x7fa8940a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0011500 tx=0x7fa8a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:00.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.187+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8a809c250 con 0x7fa8b010bfc0 2024-09-12T03:05:00.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.376+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa8b002d7c0 con 0x7fa8b010bfc0 2024-09-12T03:05:00.377 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.377+0000 7fa8acc00640 1 -- 172.21.15.57:0/3416603710 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa8a809bb50 con 0x7fa8b010bfc0 2024-09-12T03:05:00.378 INFO:teuthology.orchestra.run.smithi057.stdout:300647710760 2024-09-12T03:05:00.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 msgr2=0x7fa8940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 0x7fa8940a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0011500 tx=0x7fa8a0005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:00.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 msgr2=0x7fa8b019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 0x7fa8b019cb80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800da10 tx=0x7fa8a800ded0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.380 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 shutdown_connections 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fa8940a2c90 0x7fa8940a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa8b010bfc0 0x7fa8b019cb80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8b01034b0 0x7fa8b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 --2- 172.21.15.57:0/3416603710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8b0102b90 0x7fa8b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.379+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 >> 172.21.15.57:0/3416603710 conn(0x7fa8b00fc610 msgr2=0x7fa8b00fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.380+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 shutdown_connections 2024-09-12T03:05:00.381 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:00.380+0000 7fa8b4e00640 1 -- 172.21.15.57:0/3416603710 wait complete. 2024-09-12T03:05:00.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:00 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3416603710' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:05:00.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:00 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3416603710' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:05:00.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:00 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3416603710' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:05:01.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710756 got 300647710760 for osd.10 2024-09-12T03:05:01.222 DEBUG:teuthology.parallel:result is None 2024-09-12T03:05:01.223 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:05:01.223 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:05:01.605 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:01.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:01 smithi057 ceph-mon[30361]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:01 smithi204 ceph-mon[35013]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:01.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:01 smithi181 ceph-mon[35320]: pgmap v366: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.199+0000 7f6029200640 1 -- 172.21.15.57:0/298944667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 msgr2=0x7f60241043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.199+0000 7f6029200640 1 --2- 172.21.15.57:0/298944667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f60241043f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6014009900 tx=0x7f601402a410 comp rx=0 tx=0).stop 2024-09-12T03:05:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.199+0000 7f6029200640 1 -- 172.21.15.57:0/298944667 shutdown_connections 2024-09-12T03:05:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 --2- 172.21.15.57:0/298944667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6024109140 0x7f6024110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.201 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 --2- 172.21.15.57:0/298944667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60241049c0 0x7f6024108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 --2- 172.21.15.57:0/298944667 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f60241043f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 -- 172.21.15.57:0/298944667 >> 172.21.15.57:0/298944667 conn(0x7f60240ffa50 msgr2=0x7f6024101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 -- 172.21.15.57:0/298944667 shutdown_connections 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.200+0000 7f6029200640 1 -- 172.21.15.57:0/298944667 wait complete. 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 Processor -- start 2024-09-12T03:05:02.202 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 -- start start 2024-09-12T03:05:02.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 0x7f602419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6024109140 0x7f602419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60241141b0 con 0x7f6024104010 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6024114030 con 0x7f60241049c0 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6029200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6024114330 con 0x7f6024109140 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.201+0000 7f6021e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 0x7f602419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42804/0 (socket says 172.21.15.57:42804) 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 -- 172.21.15.57:0/1966689746 learned_addr learned my addr 172.21.15.57:0/1966689746 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6022800640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6024109140 0x7f602419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 0x7f602419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:45612/0 (socket says 172.21.15.57:45612) 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 -- 172.21.15.57:0/1966689746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6024109140 msgr2=0x7f602419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6024109140 0x7f602419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 -- 172.21.15.57:0/1966689746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 msgr2=0x7f602419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 0x7f602419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 -- 172.21.15.57:0/1966689746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602419f640 con 0x7f6024104010 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.202+0000 7f6021e00640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6014004770 tx=0x7f6014003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.203+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601403f070 con 0x7f6024104010 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.203+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6014031ca0 con 0x7f6024104010 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.203+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014035030 con 0x7f6024104010 2024-09-12T03:05:02.205 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.203+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602419f8d0 con 0x7f6024104010 2024-09-12T03:05:02.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.203+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602419fd60 con 0x7f6024104010 2024-09-12T03:05:02.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.204+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60241043f0 con 0x7f6024104010 2024-09-12T03:05:02.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.206+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6014042070 con 0x7f6024104010 2024-09-12T03:05:02.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.207+0000 7f601aa00640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 0x7f60080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.207+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f60140efc60 con 0x7f6024104010 2024-09-12T03:05:02.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.207+0000 7f6021400640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 0x7f60080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.208 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.208+0000 7f6021400640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 0x7f60080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60240ffe70 tx=0x7f6010005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:02.212 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.211+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60140bbfe0 con 0x7f6024104010 2024-09-12T03:05:02.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.393+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f602410c390 con 0x7f60080a2c10 2024-09-12T03:05:02.402 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.402+0000 7f601aa00640 1 -- 172.21.15.57:0/1966689746 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340361 (secure 0 0 0) 0x7f6010004030 con 0x7f60080a2c10 2024-09-12T03:05:02.406 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:02.408 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 msgr2=0x7f60080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 0x7f60080a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f60240ffe70 tx=0x7f6010005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 msgr2=0x7f602419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6014004770 tx=0x7f6014003c10 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 shutdown_connections 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f60080a2c10 0x7f60080a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6024109140 0x7f602419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60241049c0 0x7f602419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 --2- 172.21.15.57:0/1966689746 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6024104010 0x7f602419a690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 >> 172.21.15.57:0/1966689746 conn(0x7f60240ffa50 msgr2=0x7f6024100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:02.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 shutdown_connections 2024-09-12T03:05:02.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:02.407+0000 7f6029200640 1 -- 172.21.15.57:0/1966689746 wait complete. 2024-09-12T03:05:03.221 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-12T03:05:01.740457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657266,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":545,"num_read_kb":396,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372688,"kb_used_data":15056,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569136,"statfs":{"total":1151940427776,"available":1151558795264,"internally_reserved":0,"allocated":15417344,"data_stored":8600192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003849"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461083+0000","last_change":"2024-09-12T03:03:04.425029+0000","last_active":"2024-09-12T03:04:42.461083+0000","last_peered":"2024-09-12T03:04:42.461083+0000","last_clean":"2024-09-12T03:04:42.461083+0000","last_became_active":"2024-09-12T03:03:04.423326+0000","last_became_peered":"2024-09-12T03:03:04.423326+0000","last_unstale":"2024-09-12T03:04:42.461083+0000","last_undegraded":"2024-09-12T03:04:42.461083+0000","last_fullsized":"2024-09-12T03:04:42.461083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:28.269107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459552+0000","last_change":"2024-09-12T03:03:07.449348+0000","last_active":"2024-09-12T03:04:42.459552+0000","last_peered":"2024-09-12T03:04:42.459552+0000","last_clean":"2024-09-12T03:04:42.459552+0000","last_became_active":"2024-09-12T03:03:07.448813+0000","last_became_peered":"2024-09-12T03:03:07.448813+0000","last_unstale":"2024-09-12T03:04:42.459552+0000","last_undegraded":"2024-09-12T03:04:42.459552+0000","last_fullsized":"2024-09-12T03:04:42.459552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:31.404583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459020+0000","last_change":"2024-09-12T03:03:17.530646+0000","last_active":"2024-09-12T03:04:42.459020+0000","last_peered":"2024-09-12T03:04:42.459020+0000","last_clean":"2024-09-12T03:04:42.459020+0000","last_became_active":"2024-09-12T03:03:17.530228+0000","last_became_peered":"2024-09-12T03:03:17.530228+0000","last_unstale":"2024-09-12T03:04:42.459020+0000","last_undegraded":"2024-09-12T03:04:42.459020+0000","last_fullsized":"2024-09-12T03:04:42.459020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.763496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459744+0000","last_change":"2024-09-12T03:03:04.415539+0000","last_active":"2024-09-12T03:04:42.459744+0000","last_peered":"2024-09-12T03:04:42.459744+0000","last_clean":"2024-09-12T03:04:42.459744+0000","last_became_active":"2024-09-12T03:03:04.415163+0000","last_became_peered":"2024-09-12T03:03:04.415163+0000","last_unstale":"2024-09-12T03:04:42.459744+0000","last_undegraded":"2024-09-12T03:04:42.459744+0000","last_fullsized":"2024-09-12T03:04:42.459744+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:12.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017157+0000","last_change":"2024-09-12T03:03:07.447808+0000","last_active":"2024-09-12T03:04:43.017157+0000","last_peered":"2024-09-12T03:04:43.017157+0000","last_clean":"2024-09-12T03:04:43.017157+0000","last_became_active":"2024-09-12T03:03:07.446760+0000","last_became_peered":"2024-09-12T03:03:07.446760+0000","last_unstale":"2024-09-12T03:04:43.017157+0000","last_undegraded":"2024-09-12T03:04:43.017157+0000","last_fullsized":"2024-09-12T03:04:43.017157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:56.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850298+0000","last_change":"2024-09-12T03:03:17.530408+0000","last_active":"2024-09-12T03:04:42.850298+0000","last_peered":"2024-09-12T03:04:42.850298+0000","last_clean":"2024-09-12T03:04:42.850298+0000","last_became_active":"2024-09-12T03:03:17.530073+0000","last_became_peered":"2024-09-12T03:03:17.530073+0000","last_unstale":"2024-09-12T03:04:42.850298+0000","last_undegraded":"2024-09-12T03:04:42.850298+0000","last_fullsized":"2024-09-12T03:04:42.850298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:57.199786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462801+0000","last_change":"2024-09-12T03:03:04.422168+0000","last_active":"2024-09-12T03:04:42.462801+0000","last_peered":"2024-09-12T03:04:42.462801+0000","last_clean":"2024-09-12T03:04:42.462801+0000","last_became_active":"2024-09-12T03:03:04.421739+0000","last_became_peered":"2024-09-12T03:03:04.421739+0000","last_unstale":"2024-09-12T03:04:42.462801+0000","last_undegraded":"2024-09-12T03:04:42.462801+0000","last_fullsized":"2024-09-12T03:04:42.462801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:28.080796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064640+0000","last_change":"2024-09-12T03:03:07.445256+0000","last_active":"2024-09-12T03:04:43.064640+0000","last_peered":"2024-09-12T03:04:43.064640+0000","last_clean":"2024-09-12T03:04:43.064640+0000","last_became_active":"2024-09-12T03:03:07.444919+0000","last_became_peered":"2024-09-12T03:03:07.444919+0000","last_unstale":"2024-09-12T03:04:43.064640+0000","last_undegraded":"2024-09-12T03:04:43.064640+0000","last_fullsized":"2024-09-12T03:04:43.064640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:42.826440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850739+0000","last_change":"2024-09-12T03:03:42.456763+0000","last_active":"2024-09-12T03:04:42.850739+0000","last_peered":"2024-09-12T03:04:42.850739+0000","last_clean":"2024-09-12T03:04:42.850739+0000","last_became_active":"2024-09-12T03:03:42.456267+0000","last_became_peered":"2024-09-12T03:03:42.456267+0000","last_unstale":"2024-09-12T03:04:42.850739+0000","last_undegraded":"2024-09-12T03:04:42.850739+0000","last_fullsized":"2024-09-12T03:04:42.850739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:17.092878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462686+0000","last_change":"2024-09-12T03:03:04.423817+0000","last_active":"2024-09-12T03:04:42.462686+0000","last_peered":"2024-09-12T03:04:42.462686+0000","last_clean":"2024-09-12T03:04:42.462686+0000","last_became_active":"2024-09-12T03:03:04.422617+0000","last_became_peered":"2024-09-12T03:03:04.422617+0000","last_unstale":"2024-09-12T03:04:42.462686+0000","last_undegraded":"2024-09-12T03:04:42.462686+0000","last_fullsized":"2024-09-12T03:04:42.462686+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.309338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719320+0000","last_change":"2024-09-12T03:03:07.454768+0000","last_active":"2024-09-12T03:04:42.719320+0000","last_peered":"2024-09-12T03:04:42.719320+0000","last_clean":"2024-09-12T03:04:42.719320+0000","last_became_active":"2024-09-12T03:03:07.454414+0000","last_became_peered":"2024-09-12T03:03:07.454414+0000","last_unstale":"2024-09-12T03:04:42.719320+0000","last_undegraded":"2024-09-12T03:04:42.719320+0000","last_fullsized":"2024-09-12T03:04:42.719320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:39.968938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850377+0000","last_change":"2024-09-12T03:03:17.524984+0000","last_active":"2024-09-12T03:04:42.850377+0000","last_peered":"2024-09-12T03:04:42.850377+0000","last_clean":"2024-09-12T03:04:42.850377+0000","last_became_active":"2024-09-12T03:03:17.524516+0000","last_became_peered":"2024-09-12T03:03:17.524516+0000","last_unstale":"2024-09-12T03:04:42.850377+0000","last_undegraded":"2024-09-12T03:04:42.850377+0000","last_fullsized":"2024-09-12T03:04:42.850377+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:34.342631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458382+0000","last_change":"2024-09-12T03:03:07.450348+0000","last_active":"2024-09-12T03:04:42.458382+0000","last_peered":"2024-09-12T03:04:42.458382+0000","last_clean":"2024-09-12T03:04:42.458382+0000","last_became_active":"2024-09-12T03:03:07.448968+0000","last_became_peered":"2024-09-12T03:03:07.448968+0000","last_unstale":"2024-09-12T03:04:42.458382+0000","last_undegraded":"2024-09-12T03:04:42.458382+0000","last_fullsized":"2024-09-12T03:04:42.458382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:19.053409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463637+0000","last_change":"2024-09-12T03:03:41.676526+0000","last_active":"2024-09-12T03:04:42.463637+0000","last_peered":"2024-09-12T03:04:42.463637+0000","last_clean":"2024-09-12T03:04:42.463637+0000","last_became_active":"2024-09-12T03:03:41.673301+0000","last_became_peered":"2024-09-12T03:03:41.673301+0000","last_unstale":"2024-09-12T03:04:42.463637+0000","last_undegraded":"2024-09-12T03:04:42.463637+0000","last_fullsized":"2024-09-12T03:04:42.463637+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:04.648668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850619+0000","last_change":"2024-09-12T03:03:17.526846+0000","last_active":"2024-09-12T03:04:42.850619+0000","last_peered":"2024-09-12T03:04:42.850619+0000","last_clean":"2024-09-12T03:04:42.850619+0000","last_became_active":"2024-09-12T03:03:17.526412+0000","last_became_peered":"2024-09-12T03:03:17.526412+0000","last_unstale":"2024-09-12T03:04:42.850619+0000","last_undegraded":"2024-09-12T03:04:42.850619+0000","last_fullsized":"2024-09-12T03:04:42.850619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458566+0000","last_change":"2024-09-12T03:03:07.453989+0000","last_active":"2024-09-12T03:04:42.458566+0000","last_peered":"2024-09-12T03:04:42.458566+0000","last_clean":"2024-09-12T03:04:42.458566+0000","last_became_active":"2024-09-12T03:03:07.451796+0000","last_became_peered":"2024-09-12T03:03:07.451796+0000","last_unstale":"2024-09-12T03:04:42.458566+0000","last_undegraded":"2024-09-12T03:04:42.458566+0000","last_fullsized":"2024-09-12T03:04:42.458566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:27.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459110+0000","last_change":"2024-09-12T03:03:04.423756+0000","last_active":"2024-09-12T03:04:42.459110+0000","last_peered":"2024-09-12T03:04:42.459110+0000","last_clean":"2024-09-12T03:04:42.459110+0000","last_became_active":"2024-09-12T03:03:04.423075+0000","last_became_peered":"2024-09-12T03:03:04.423075+0000","last_unstale":"2024-09-12T03:04:42.459110+0000","last_undegraded":"2024-09-12T03:04:42.459110+0000","last_fullsized":"2024-09-12T03:04:42.459110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.201739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850984+0000","last_change":"2024-09-12T03:03:17.532906+0000","last_active":"2024-09-12T03:04:42.850984+0000","last_peered":"2024-09-12T03:04:42.850984+0000","last_clean":"2024-09-12T03:04:42.850984+0000","last_became_active":"2024-09-12T03:03:17.532343+0000","last_became_peered":"2024-09-12T03:03:17.532343+0000","last_unstale":"2024-09-12T03:04:42.850984+0000","last_undegraded":"2024-09-12T03:04:42.850984+0000","last_fullsized":"2024-09-12T03:04:42.850984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:05.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463613+0000","last_change":"2024-09-12T03:03:07.450375+0000","last_active":"2024-09-12T03:04:42.463613+0000","last_peered":"2024-09-12T03:04:42.463613+0000","last_clean":"2024-09-12T03:04:42.463613+0000","last_became_active":"2024-09-12T03:03:07.450061+0000","last_became_peered":"2024-09-12T03:03:07.450061+0000","last_unstale":"2024-09-12T03:04:42.463613+0000","last_undegraded":"2024-09-12T03:04:42.463613+0000","last_fullsized":"2024-09-12T03:04:42.463613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:05.922237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458364+0000","last_change":"2024-09-12T03:03:04.423664+0000","last_active":"2024-09-12T03:04:42.458364+0000","last_peered":"2024-09-12T03:04:42.458364+0000","last_clean":"2024-09-12T03:04:42.458364+0000","last_became_active":"2024-09-12T03:03:04.423285+0000","last_became_peered":"2024-09-12T03:03:04.423285+0000","last_unstale":"2024-09-12T03:04:42.458364+0000","last_undegraded":"2024-09-12T03:04:42.458364+0000","last_fullsized":"2024-09-12T03:04:42.458364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.782484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460713+0000","last_change":"2024-09-12T03:03:17.535385+0000","last_active":"2024-09-12T03:04:42.460713+0000","last_peered":"2024-09-12T03:04:42.460713+0000","last_clean":"2024-09-12T03:04:42.460713+0000","last_became_active":"2024-09-12T03:03:17.534775+0000","last_became_peered":"2024-09-12T03:03:17.534775+0000","last_unstale":"2024-09-12T03:04:42.460713+0000","last_undegraded":"2024-09-12T03:04:42.460713+0000","last_fullsized":"2024-09-12T03:04:42.460713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:02.549462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462139+0000","last_change":"2024-09-12T03:03:07.442786+0000","last_active":"2024-09-12T03:04:42.462139+0000","last_peered":"2024-09-12T03:04:42.462139+0000","last_clean":"2024-09-12T03:04:42.462139+0000","last_became_active":"2024-09-12T03:03:07.442456+0000","last_became_peered":"2024-09-12T03:03:07.442456+0000","last_unstale":"2024-09-12T03:04:42.462139+0000","last_undegraded":"2024-09-12T03:04:42.462139+0000","last_fullsized":"2024-09-12T03:04:42.462139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.882361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064499+0000","last_change":"2024-09-12T03:03:41.675913+0000","last_active":"2024-09-12T03:04:43.064499+0000","last_peered":"2024-09-12T03:04:43.064499+0000","last_clean":"2024-09-12T03:04:43.064499+0000","last_became_active":"2024-09-12T03:03:41.675434+0000","last_became_peered":"2024-09-12T03:03:41.675434+0000","last_unstale":"2024-09-12T03:04:43.064499+0000","last_undegraded":"2024-09-12T03:04:43.064499+0000","last_fullsized":"2024-09-12T03:04:43.064499+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:36.484404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850833+0000","last_change":"2024-09-12T03:03:42.177089+0000","last_active":"2024-09-12T03:04:42.850833+0000","last_peered":"2024-09-12T03:04:42.850833+0000","last_clean":"2024-09-12T03:04:42.850833+0000","last_became_active":"2024-09-12T03:03:42.176602+0000","last_became_peered":"2024-09-12T03:03:42.176602+0000","last_unstale":"2024-09-12T03:04:42.850833+0000","last_undegraded":"2024-09-12T03:04:42.850833+0000","last_fullsized":"2024-09-12T03:04:42.850833+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:04.336826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017043+0000","last_change":"2024-09-12T03:03:07.457972+0000","last_active":"2024-09-12T03:04:43.017043+0000","last_peered":"2024-09-12T03:04:43.017043+0000","last_clean":"2024-09-12T03:04:43.017043+0000","last_became_active":"2024-09-12T03:03:07.457681+0000","last_became_peered":"2024-09-12T03:03:07.457681+0000","last_unstale":"2024-09-12T03:04:43.017043+0000","last_undegraded":"2024-09-12T03:04:43.017043+0000","last_fullsized":"2024-09-12T03:04:43.017043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:26.733432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850887+0000","last_change":"2024-09-12T03:03:04.629721+0000","last_active":"2024-09-12T03:04:42.850887+0000","last_peered":"2024-09-12T03:04:42.850887+0000","last_clean":"2024-09-12T03:04:42.850887+0000","last_became_active":"2024-09-12T03:03:04.629362+0000","last_became_peered":"2024-09-12T03:03:04.629362+0000","last_unstale":"2024-09-12T03:04:42.850887+0000","last_undegraded":"2024-09-12T03:04:42.850887+0000","last_fullsized":"2024-09-12T03:04:42.850887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:09.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459216+0000","last_change":"2024-09-12T03:03:17.521048+0000","last_active":"2024-09-12T03:04:42.459216+0000","last_peered":"2024-09-12T03:04:42.459216+0000","last_clean":"2024-09-12T03:04:42.459216+0000","last_became_active":"2024-09-12T03:03:17.520640+0000","last_became_peered":"2024-09-12T03:03:17.520640+0000","last_unstale":"2024-09-12T03:04:42.459216+0000","last_undegraded":"2024-09-12T03:04:42.459216+0000","last_fullsized":"2024-09-12T03:04:42.459216+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:06.243255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459661+0000","last_change":"2024-09-12T03:03:07.445497+0000","last_active":"2024-09-12T03:04:42.459661+0000","last_peered":"2024-09-12T03:04:42.459661+0000","last_clean":"2024-09-12T03:04:42.459661+0000","last_became_active":"2024-09-12T03:03:07.442245+0000","last_became_peered":"2024-09-12T03:03:07.442245+0000","last_unstale":"2024-09-12T03:04:42.459661+0000","last_undegraded":"2024-09-12T03:04:42.459661+0000","last_fullsized":"2024-09-12T03:04:42.459661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:13.147572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461081+0000","last_change":"2024-09-12T03:03:04.424438+0000","last_active":"2024-09-12T03:04:42.461081+0000","last_peered":"2024-09-12T03:04:42.461081+0000","last_clean":"2024-09-12T03:04:42.461081+0000","last_became_active":"2024-09-12T03:03:04.424078+0000","last_became_peered":"2024-09-12T03:03:04.424078+0000","last_unstale":"2024-09-12T03:04:42.461081+0000","last_undegraded":"2024-09-12T03:04:42.461081+0000","last_fullsized":"2024-09-12T03:04:42.461081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:33.090082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457838+0000","last_change":"2024-09-12T03:03:17.532767+0000","last_active":"2024-09-12T03:04:42.457838+0000","last_peered":"2024-09-12T03:04:42.457838+0000","last_clean":"2024-09-12T03:04:42.457838+0000","last_became_active":"2024-09-12T03:03:17.532408+0000","last_became_peered":"2024-09-12T03:03:17.532408+0000","last_unstale":"2024-09-12T03:04:42.457838+0000","last_undegraded":"2024-09-12T03:04:42.457838+0000","last_fullsized":"2024-09-12T03:04:42.457838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:29.294605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884833+0000","last_change":"2024-09-12T03:03:07.448931+0000","last_active":"2024-09-12T03:04:42.884833+0000","last_peered":"2024-09-12T03:04:42.884833+0000","last_clean":"2024-09-12T03:04:42.884833+0000","last_became_active":"2024-09-12T03:03:07.447204+0000","last_became_peered":"2024-09-12T03:03:07.447204+0000","last_unstale":"2024-09-12T03:04:42.884833+0000","last_undegraded":"2024-09-12T03:04:42.884833+0000","last_fullsized":"2024-09-12T03:04:42.884833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:10.491591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462929+0000","last_change":"2024-09-12T03:03:04.425805+0000","last_active":"2024-09-12T03:04:42.462929+0000","last_peered":"2024-09-12T03:04:42.462929+0000","last_clean":"2024-09-12T03:04:42.462929+0000","last_became_active":"2024-09-12T03:03:04.425469+0000","last_became_peered":"2024-09-12T03:03:04.425469+0000","last_unstale":"2024-09-12T03:04:42.462929+0000","last_undegraded":"2024-09-12T03:04:42.462929+0000","last_fullsized":"2024-09-12T03:04:42.462929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:27.976934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463678+0000","last_change":"2024-09-12T03:03:17.525537+0000","last_active":"2024-09-12T03:04:42.463678+0000","last_peered":"2024-09-12T03:04:42.463678+0000","last_clean":"2024-09-12T03:04:42.463678+0000","last_became_active":"2024-09-12T03:03:17.525030+0000","last_became_peered":"2024-09-12T03:03:17.525030+0000","last_unstale":"2024-09-12T03:04:42.463678+0000","last_undegraded":"2024-09-12T03:04:42.463678+0000","last_fullsized":"2024-09-12T03:04:42.463678+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:53.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542918+0000","last_change":"2024-09-12T03:03:07.443712+0000","last_active":"2024-09-12T03:04:42.542918+0000","last_peered":"2024-09-12T03:04:42.542918+0000","last_clean":"2024-09-12T03:04:42.542918+0000","last_became_active":"2024-09-12T03:03:07.443374+0000","last_became_peered":"2024-09-12T03:03:07.443374+0000","last_unstale":"2024-09-12T03:04:42.542918+0000","last_undegraded":"2024-09-12T03:04:42.542918+0000","last_fullsized":"2024-09-12T03:04:42.542918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:09.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461477+0000","last_change":"2024-09-12T03:03:04.427683+0000","last_active":"2024-09-12T03:04:42.461477+0000","last_peered":"2024-09-12T03:04:42.461477+0000","last_clean":"2024-09-12T03:04:42.461477+0000","last_became_active":"2024-09-12T03:03:04.427276+0000","last_became_peered":"2024-09-12T03:03:04.427276+0000","last_unstale":"2024-09-12T03:04:42.461477+0000","last_undegraded":"2024-09-12T03:04:42.461477+0000","last_fullsized":"2024-09-12T03:04:42.461477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:44.540369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462141+0000","last_change":"2024-09-12T03:03:41.878110+0000","last_active":"2024-09-12T03:04:42.462141+0000","last_peered":"2024-09-12T03:04:42.462141+0000","last_clean":"2024-09-12T03:04:42.462141+0000","last_became_active":"2024-09-12T03:03:41.877719+0000","last_became_peered":"2024-09-12T03:03:41.877719+0000","last_unstale":"2024-09-12T03:04:42.462141+0000","last_undegraded":"2024-09-12T03:04:42.462141+0000","last_fullsized":"2024-09-12T03:04:42.462141+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:08.312660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542816+0000","last_change":"2024-09-12T03:03:07.454093+0000","last_active":"2024-09-12T03:04:42.542816+0000","last_peered":"2024-09-12T03:04:42.542816+0000","last_clean":"2024-09-12T03:04:42.542816+0000","last_became_active":"2024-09-12T03:03:07.453676+0000","last_became_peered":"2024-09-12T03:03:07.453676+0000","last_unstale":"2024-09-12T03:04:42.542816+0000","last_undegraded":"2024-09-12T03:04:42.542816+0000","last_fullsized":"2024-09-12T03:04:42.542816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:18.045572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458590+0000","last_change":"2024-09-12T03:03:04.425336+0000","last_active":"2024-09-12T03:04:42.458590+0000","last_peered":"2024-09-12T03:04:42.458590+0000","last_clean":"2024-09-12T03:04:42.458590+0000","last_became_active":"2024-09-12T03:03:04.424890+0000","last_became_peered":"2024-09-12T03:03:04.424890+0000","last_unstale":"2024-09-12T03:04:42.458590+0000","last_undegraded":"2024-09-12T03:04:42.458590+0000","last_fullsized":"2024-09-12T03:04:42.458590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.443627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884698+0000","last_change":"2024-09-12T03:03:17.535300+0000","last_active":"2024-09-12T03:04:42.884698+0000","last_peered":"2024-09-12T03:04:42.884698+0000","last_clean":"2024-09-12T03:04:42.884698+0000","last_became_active":"2024-09-12T03:03:17.534937+0000","last_became_peered":"2024-09-12T03:03:17.534937+0000","last_unstale":"2024-09-12T03:04:42.884698+0000","last_undegraded":"2024-09-12T03:04:42.884698+0000","last_fullsized":"2024-09-12T03:04:42.884698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:54.874973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718723+0000","last_change":"2024-09-12T03:03:07.453964+0000","last_active":"2024-09-12T03:04:42.718723+0000","last_peered":"2024-09-12T03:04:42.718723+0000","last_clean":"2024-09-12T03:04:42.718723+0000","last_became_active":"2024-09-12T03:03:07.453622+0000","last_became_peered":"2024-09-12T03:03:07.453622+0000","last_unstale":"2024-09-12T03:04:42.718723+0000","last_undegraded":"2024-09-12T03:04:42.718723+0000","last_fullsized":"2024-09-12T03:04:42.718723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:17.452519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":112,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.914281+0000","last_change":"2024-09-12T03:03:04.428526+0000","last_active":"2024-09-12T03:04:59.914281+0000","last_peered":"2024-09-12T03:04:59.914281+0000","last_clean":"2024-09-12T03:04:59.914281+0000","last_became_active":"2024-09-12T03:03:04.428221+0000","last_became_peered":"2024-09-12T03:03:04.428221+0000","last_unstale":"2024-09-12T03:04:59.914281+0000","last_undegraded":"2024-09-12T03:04:59.914281+0000","last_fullsized":"2024-09-12T03:04:59.914281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:50.999346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884444+0000","last_change":"2024-09-12T03:03:17.535146+0000","last_active":"2024-09-12T03:04:42.884444+0000","last_peered":"2024-09-12T03:04:42.884444+0000","last_clean":"2024-09-12T03:04:42.884444+0000","last_became_active":"2024-09-12T03:03:17.534677+0000","last_became_peered":"2024-09-12T03:03:17.534677+0000","last_unstale":"2024-09-12T03:04:42.884444+0000","last_undegraded":"2024-09-12T03:04:42.884444+0000","last_fullsized":"2024-09-12T03:04:42.884444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:09.874993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884420+0000","last_change":"2024-09-12T03:03:07.431348+0000","last_active":"2024-09-12T03:04:42.884420+0000","last_peered":"2024-09-12T03:04:42.884420+0000","last_clean":"2024-09-12T03:04:42.884420+0000","last_became_active":"2024-09-12T03:03:07.430912+0000","last_became_peered":"2024-09-12T03:03:07.430912+0000","last_unstale":"2024-09-12T03:04:42.884420+0000","last_undegraded":"2024-09-12T03:04:42.884420+0000","last_fullsized":"2024-09-12T03:04:42.884420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:27.644324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458256+0000","last_change":"2024-09-12T03:03:42.456245+0000","last_active":"2024-09-12T03:04:42.458256+0000","last_peered":"2024-09-12T03:04:42.458256+0000","last_clean":"2024-09-12T03:04:42.458256+0000","last_became_active":"2024-09-12T03:03:42.455812+0000","last_became_peered":"2024-09-12T03:03:42.455812+0000","last_unstale":"2024-09-12T03:04:42.458256+0000","last_undegraded":"2024-09-12T03:04:42.458256+0000","last_fullsized":"2024-09-12T03:04:42.458256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719073+0000","last_change":"2024-09-12T03:03:17.527371+0000","last_active":"2024-09-12T03:04:42.719073+0000","last_peered":"2024-09-12T03:04:42.719073+0000","last_clean":"2024-09-12T03:04:42.719073+0000","last_became_active":"2024-09-12T03:03:17.526818+0000","last_became_peered":"2024-09-12T03:03:17.526818+0000","last_unstale":"2024-09-12T03:04:42.719073+0000","last_undegraded":"2024-09-12T03:04:42.719073+0000","last_fullsized":"2024-09-12T03:04:42.719073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:49.519942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458178+0000","last_change":"2024-09-12T03:03:07.451364+0000","last_active":"2024-09-12T03:04:42.458178+0000","last_peered":"2024-09-12T03:04:42.458178+0000","last_clean":"2024-09-12T03:04:42.458178+0000","last_became_active":"2024-09-12T03:03:07.450964+0000","last_became_peered":"2024-09-12T03:03:07.450964+0000","last_unstale":"2024-09-12T03:04:42.458178+0000","last_undegraded":"2024-09-12T03:04:42.458178+0000","last_fullsized":"2024-09-12T03:04:42.458178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850880+0000","last_change":"2024-09-12T03:03:04.430485+0000","last_active":"2024-09-12T03:04:42.850880+0000","last_peered":"2024-09-12T03:04:42.850880+0000","last_clean":"2024-09-12T03:04:42.850880+0000","last_became_active":"2024-09-12T03:03:04.430096+0000","last_became_peered":"2024-09-12T03:03:04.430096+0000","last_unstale":"2024-09-12T03:04:42.850880+0000","last_undegraded":"2024-09-12T03:04:42.850880+0000","last_fullsized":"2024-09-12T03:04:42.850880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:26.486223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458445+0000","last_change":"2024-09-12T03:01:48.369350+0000","last_active":"2024-09-12T03:04:42.458445+0000","last_peered":"2024-09-12T03:04:42.458445+0000","last_clean":"2024-09-12T03:04:42.458445+0000","last_became_active":"2024-09-12T03:01:48.369011+0000","last_became_peered":"2024-09-12T03:01:48.369011+0000","last_unstale":"2024-09-12T03:04:42.458445+0000","last_undegraded":"2024-09-12T03:04:42.458445+0000","last_fullsized":"2024-09-12T03:04:42.458445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_clean_scrub_stamp":"2024-09-12T02:58:06.597228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:57.348343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458778+0000","last_change":"2024-09-12T03:03:17.541157+0000","last_active":"2024-09-12T03:04:42.458778+0000","last_peered":"2024-09-12T03:04:42.458778+0000","last_clean":"2024-09-12T03:04:42.458778+0000","last_became_active":"2024-09-12T03:03:17.540687+0000","last_became_peered":"2024-09-12T03:03:17.540687+0000","last_unstale":"2024-09-12T03:04:42.458778+0000","last_undegraded":"2024-09-12T03:04:42.458778+0000","last_fullsized":"2024-09-12T03:04:42.458778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:35.072323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458920+0000","last_change":"2024-09-12T03:03:07.445674+0000","last_active":"2024-09-12T03:04:42.458920+0000","last_peered":"2024-09-12T03:04:42.458920+0000","last_clean":"2024-09-12T03:04:42.458920+0000","last_became_active":"2024-09-12T03:03:07.445237+0000","last_became_peered":"2024-09-12T03:03:07.445237+0000","last_unstale":"2024-09-12T03:04:42.458920+0000","last_undegraded":"2024-09-12T03:04:42.458920+0000","last_fullsized":"2024-09-12T03:04:42.458920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:56.876238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459575+0000","last_change":"2024-09-12T03:03:04.629794+0000","last_active":"2024-09-12T03:04:42.459575+0000","last_peered":"2024-09-12T03:04:42.459575+0000","last_clean":"2024-09-12T03:04:42.459575+0000","last_became_active":"2024-09-12T03:03:04.629352+0000","last_became_peered":"2024-09-12T03:03:04.629352+0000","last_unstale":"2024-09-12T03:04:42.459575+0000","last_undegraded":"2024-09-12T03:04:42.459575+0000","last_fullsized":"2024-09-12T03:04:42.459575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:32.168567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064374+0000","last_change":"2024-09-12T03:03:17.525419+0000","last_active":"2024-09-12T03:04:43.064374+0000","last_peered":"2024-09-12T03:04:43.064374+0000","last_clean":"2024-09-12T03:04:43.064374+0000","last_became_active":"2024-09-12T03:03:17.524894+0000","last_became_peered":"2024-09-12T03:03:17.524894+0000","last_unstale":"2024-09-12T03:04:43.064374+0000","last_undegraded":"2024-09-12T03:04:43.064374+0000","last_fullsized":"2024-09-12T03:04:43.064374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:57.441301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458742+0000","last_change":"2024-09-12T03:03:07.447990+0000","last_active":"2024-09-12T03:04:42.458742+0000","last_peered":"2024-09-12T03:04:42.458742+0000","last_clean":"2024-09-12T03:04:42.458742+0000","last_became_active":"2024-09-12T03:03:07.447648+0000","last_became_peered":"2024-09-12T03:03:07.447648+0000","last_unstale":"2024-09-12T03:04:42.458742+0000","last_undegraded":"2024-09-12T03:04:42.458742+0000","last_fullsized":"2024-09-12T03:04:42.458742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:48.639273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719564+0000","last_change":"2024-09-12T03:03:04.421306+0000","last_active":"2024-09-12T03:04:42.719564+0000","last_peered":"2024-09-12T03:04:42.719564+0000","last_clean":"2024-09-12T03:04:42.719564+0000","last_became_active":"2024-09-12T03:03:04.420688+0000","last_became_peered":"2024-09-12T03:03:04.420688+0000","last_unstale":"2024-09-12T03:04:42.719564+0000","last_undegraded":"2024-09-12T03:04:42.719564+0000","last_fullsized":"2024-09-12T03:04:42.719564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:15.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016756+0000","last_change":"2024-09-12T03:03:17.530457+0000","last_active":"2024-09-12T03:04:43.016756+0000","last_peered":"2024-09-12T03:04:43.016756+0000","last_clean":"2024-09-12T03:04:43.016756+0000","last_became_active":"2024-09-12T03:03:17.530044+0000","last_became_peered":"2024-09-12T03:03:17.530044+0000","last_unstale":"2024-09-12T03:04:43.016756+0000","last_undegraded":"2024-09-12T03:04:43.016756+0000","last_fullsized":"2024-09-12T03:04:43.016756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:01.304089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459018+0000","last_change":"2024-09-12T03:03:07.445935+0000","last_active":"2024-09-12T03:04:42.459018+0000","last_peered":"2024-09-12T03:04:42.459018+0000","last_clean":"2024-09-12T03:04:42.459018+0000","last_became_active":"2024-09-12T03:03:07.444960+0000","last_became_peered":"2024-09-12T03:03:07.444960+0000","last_unstale":"2024-09-12T03:04:42.459018+0000","last_undegraded":"2024-09-12T03:04:42.459018+0000","last_fullsized":"2024-09-12T03:04:42.459018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:53.246923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884822+0000","last_change":"2024-09-12T03:03:04.424147+0000","last_active":"2024-09-12T03:04:42.884822+0000","last_peered":"2024-09-12T03:04:42.884822+0000","last_clean":"2024-09-12T03:04:42.884822+0000","last_became_active":"2024-09-12T03:03:04.423787+0000","last_became_peered":"2024-09-12T03:03:04.423787+0000","last_unstale":"2024-09-12T03:04:42.884822+0000","last_undegraded":"2024-09-12T03:04:42.884822+0000","last_fullsized":"2024-09-12T03:04:42.884822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:48.155556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718464+0000","last_change":"2024-09-12T03:03:17.520974+0000","last_active":"2024-09-12T03:04:42.718464+0000","last_peered":"2024-09-12T03:04:42.718464+0000","last_clean":"2024-09-12T03:04:42.718464+0000","last_became_active":"2024-09-12T03:03:17.520332+0000","last_became_peered":"2024-09-12T03:03:17.520332+0000","last_unstale":"2024-09-12T03:04:42.718464+0000","last_undegraded":"2024-09-12T03:04:42.718464+0000","last_fullsized":"2024-09-12T03:04:42.718464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:45.061197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464537+0000","last_change":"2024-09-12T03:03:42.456589+0000","last_active":"2024-09-12T03:04:42.464537+0000","last_peered":"2024-09-12T03:04:42.464537+0000","last_clean":"2024-09-12T03:04:42.464537+0000","last_became_active":"2024-09-12T03:03:42.456147+0000","last_became_peered":"2024-09-12T03:03:42.456147+0000","last_unstale":"2024-09-12T03:04:42.464537+0000","last_undegraded":"2024-09-12T03:04:42.464537+0000","last_fullsized":"2024-09-12T03:04:42.464537+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:58.009928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463226+0000","last_change":"2024-09-12T03:03:04.420471+0000","last_active":"2024-09-12T03:04:42.463226+0000","last_peered":"2024-09-12T03:04:42.463226+0000","last_clean":"2024-09-12T03:04:42.463226+0000","last_became_active":"2024-09-12T03:03:04.420093+0000","last_became_peered":"2024-09-12T03:03:04.420093+0000","last_unstale":"2024-09-12T03:04:42.463226+0000","last_undegraded":"2024-09-12T03:04:42.463226+0000","last_fullsized":"2024-09-12T03:04:42.463226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:23.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718669+0000","last_change":"2024-09-12T03:03:17.528073+0000","last_active":"2024-09-12T03:04:42.718669+0000","last_peered":"2024-09-12T03:04:42.718669+0000","last_clean":"2024-09-12T03:04:42.718669+0000","last_became_active":"2024-09-12T03:03:17.527654+0000","last_became_peered":"2024-09-12T03:03:17.527654+0000","last_unstale":"2024-09-12T03:04:42.718669+0000","last_undegraded":"2024-09-12T03:04:42.718669+0000","last_fullsized":"2024-09-12T03:04:42.718669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:45.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718749+0000","last_change":"2024-09-12T03:03:07.455305+0000","last_active":"2024-09-12T03:04:42.718749+0000","last_peered":"2024-09-12T03:04:42.718749+0000","last_clean":"2024-09-12T03:04:42.718749+0000","last_became_active":"2024-09-12T03:03:07.454957+0000","last_became_peered":"2024-09-12T03:03:07.454957+0000","last_unstale":"2024-09-12T03:04:42.718749+0000","last_undegraded":"2024-09-12T03:04:42.718749+0000","last_fullsized":"2024-09-12T03:04:42.718749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:07.152100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462091+0000","last_change":"2024-09-12T03:03:04.424531+0000","last_active":"2024-09-12T03:04:42.462091+0000","last_peered":"2024-09-12T03:04:42.462091+0000","last_clean":"2024-09-12T03:04:42.462091+0000","last_became_active":"2024-09-12T03:03:04.424230+0000","last_became_peered":"2024-09-12T03:03:04.424230+0000","last_unstale":"2024-09-12T03:04:42.462091+0000","last_undegraded":"2024-09-12T03:04:42.462091+0000","last_fullsized":"2024-09-12T03:04:42.462091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:01.392262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884779+0000","last_change":"2024-09-12T03:03:17.532131+0000","last_active":"2024-09-12T03:04:42.884779+0000","last_peered":"2024-09-12T03:04:42.884779+0000","last_clean":"2024-09-12T03:04:42.884779+0000","last_became_active":"2024-09-12T03:03:17.531711+0000","last_became_peered":"2024-09-12T03:03:17.531711+0000","last_unstale":"2024-09-12T03:04:42.884779+0000","last_undegraded":"2024-09-12T03:04:42.884779+0000","last_fullsized":"2024-09-12T03:04:42.884779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:15.880954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458191+0000","last_change":"2024-09-12T03:03:07.449101+0000","last_active":"2024-09-12T03:04:42.458191+0000","last_peered":"2024-09-12T03:04:42.458191+0000","last_clean":"2024-09-12T03:04:42.458191+0000","last_became_active":"2024-09-12T03:03:07.448719+0000","last_became_peered":"2024-09-12T03:03:07.448719+0000","last_unstale":"2024-09-12T03:04:42.458191+0000","last_undegraded":"2024-09-12T03:04:42.458191+0000","last_fullsized":"2024-09-12T03:04:42.458191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:45.400939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:54.905444+0000","last_change":"2024-09-12T03:03:04.422174+0000","last_active":"2024-09-12T03:04:54.905444+0000","last_peered":"2024-09-12T03:04:54.905444+0000","last_clean":"2024-09-12T03:04:54.905444+0000","last_became_active":"2024-09-12T03:03:04.421771+0000","last_became_peered":"2024-09-12T03:03:04.421771+0000","last_unstale":"2024-09-12T03:04:54.905444+0000","last_undegraded":"2024-09-12T03:04:54.905444+0000","last_fullsized":"2024-09-12T03:04:54.905444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:38.149590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460718+0000","last_change":"2024-09-12T03:03:17.535123+0000","last_active":"2024-09-12T03:04:42.460718+0000","last_peered":"2024-09-12T03:04:42.460718+0000","last_clean":"2024-09-12T03:04:42.460718+0000","last_became_active":"2024-09-12T03:03:17.534218+0000","last_became_peered":"2024-09-12T03:03:17.534218+0000","last_unstale":"2024-09-12T03:04:42.460718+0000","last_undegraded":"2024-09-12T03:04:42.460718+0000","last_fullsized":"2024-09-12T03:04:42.460718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:55.870690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458503+0000","last_change":"2024-09-12T03:03:07.453736+0000","last_active":"2024-09-12T03:04:42.458503+0000","last_peered":"2024-09-12T03:04:42.458503+0000","last_clean":"2024-09-12T03:04:42.458503+0000","last_became_active":"2024-09-12T03:03:07.453404+0000","last_became_peered":"2024-09-12T03:03:07.453404+0000","last_unstale":"2024-09-12T03:04:42.458503+0000","last_undegraded":"2024-09-12T03:04:42.458503+0000","last_fullsized":"2024-09-12T03:04:42.458503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:57.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719393+0000","last_change":"2024-09-12T03:03:04.419663+0000","last_active":"2024-09-12T03:04:42.719393+0000","last_peered":"2024-09-12T03:04:42.719393+0000","last_clean":"2024-09-12T03:04:42.719393+0000","last_became_active":"2024-09-12T03:03:04.419119+0000","last_became_peered":"2024-09-12T03:03:04.419119+0000","last_unstale":"2024-09-12T03:04:42.719393+0000","last_undegraded":"2024-09-12T03:04:42.719393+0000","last_fullsized":"2024-09-12T03:04:42.719393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:11.809683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463824+0000","last_change":"2024-09-12T03:03:17.532686+0000","last_active":"2024-09-12T03:04:42.463824+0000","last_peered":"2024-09-12T03:04:42.463824+0000","last_clean":"2024-09-12T03:04:42.463824+0000","last_became_active":"2024-09-12T03:03:17.532234+0000","last_became_peered":"2024-09-12T03:03:17.532234+0000","last_unstale":"2024-09-12T03:04:42.463824+0000","last_undegraded":"2024-09-12T03:04:42.463824+0000","last_fullsized":"2024-09-12T03:04:42.463824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:49.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884316+0000","last_change":"2024-09-12T03:03:07.441911+0000","last_active":"2024-09-12T03:04:42.884316+0000","last_peered":"2024-09-12T03:04:42.884316+0000","last_clean":"2024-09-12T03:04:42.884316+0000","last_became_active":"2024-09-12T03:03:07.441323+0000","last_became_peered":"2024-09-12T03:03:07.441323+0000","last_unstale":"2024-09-12T03:04:42.884316+0000","last_undegraded":"2024-09-12T03:04:42.884316+0000","last_fullsized":"2024-09-12T03:04:42.884316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:03.830843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459398+0000","last_change":"2024-09-12T03:03:04.423648+0000","last_active":"2024-09-12T03:04:42.459398+0000","last_peered":"2024-09-12T03:04:42.459398+0000","last_clean":"2024-09-12T03:04:42.459398+0000","last_became_active":"2024-09-12T03:03:04.423274+0000","last_became_peered":"2024-09-12T03:03:04.423274+0000","last_unstale":"2024-09-12T03:04:42.459398+0000","last_undegraded":"2024-09-12T03:04:42.459398+0000","last_fullsized":"2024-09-12T03:04:42.459398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:50.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719426+0000","last_change":"2024-09-12T03:03:17.522286+0000","last_active":"2024-09-12T03:04:42.719426+0000","last_peered":"2024-09-12T03:04:42.719426+0000","last_clean":"2024-09-12T03:04:42.719426+0000","last_became_active":"2024-09-12T03:03:17.521909+0000","last_became_peered":"2024-09-12T03:03:17.521909+0000","last_unstale":"2024-09-12T03:04:42.719426+0000","last_undegraded":"2024-09-12T03:04:42.719426+0000","last_fullsized":"2024-09-12T03:04:42.719426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:13.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462495+0000","last_change":"2024-09-12T03:03:07.445445+0000","last_active":"2024-09-12T03:04:42.462495+0000","last_peered":"2024-09-12T03:04:42.462495+0000","last_clean":"2024-09-12T03:04:42.462495+0000","last_became_active":"2024-09-12T03:03:07.445086+0000","last_became_peered":"2024-09-12T03:03:07.445086+0000","last_unstale":"2024-09-12T03:04:42.462495+0000","last_undegraded":"2024-09-12T03:04:42.462495+0000","last_fullsized":"2024-09-12T03:04:42.462495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:01.046168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'32","reported_seq":212,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.902976+0000","last_change":"2024-09-12T03:03:04.425545+0000","last_active":"2024-09-12T03:04:59.902976+0000","last_peered":"2024-09-12T03:04:59.902976+0000","last_clean":"2024-09-12T03:04:59.902976+0000","last_became_active":"2024-09-12T03:03:04.425141+0000","last_became_peered":"2024-09-12T03:03:04.425141+0000","last_unstale":"2024-09-12T03:04:59.902976+0000","last_undegraded":"2024-09-12T03:04:59.902976+0000","last_fullsized":"2024-09-12T03:04:59.902976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:54.548787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62661,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":136,"num_read_kb":129,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460282+0000","last_change":"2024-09-12T03:03:17.538157+0000","last_active":"2024-09-12T03:04:42.460282+0000","last_peered":"2024-09-12T03:04:42.460282+0000","last_clean":"2024-09-12T03:04:42.460282+0000","last_became_active":"2024-09-12T03:03:17.537194+0000","last_became_peered":"2024-09-12T03:03:17.537194+0000","last_unstale":"2024-09-12T03:04:42.460282+0000","last_undegraded":"2024-09-12T03:04:42.460282+0000","last_fullsized":"2024-09-12T03:04:42.460282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:00.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064669+0000","last_change":"2024-09-12T03:03:07.446470+0000","last_active":"2024-09-12T03:04:43.064669+0000","last_peered":"2024-09-12T03:04:43.064669+0000","last_clean":"2024-09-12T03:04:43.064669+0000","last_became_active":"2024-09-12T03:03:07.445975+0000","last_became_peered":"2024-09-12T03:03:07.445975+0000","last_unstale":"2024-09-12T03:04:43.064669+0000","last_undegraded":"2024-09-12T03:04:43.064669+0000","last_fullsized":"2024-09-12T03:04:43.064669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:47.795117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719517+0000","last_change":"2024-09-12T03:03:04.421833+0000","last_active":"2024-09-12T03:04:42.719517+0000","last_peered":"2024-09-12T03:04:42.719517+0000","last_clean":"2024-09-12T03:04:42.719517+0000","last_became_active":"2024-09-12T03:03:04.421439+0000","last_became_peered":"2024-09-12T03:03:04.421439+0000","last_unstale":"2024-09-12T03:04:42.719517+0000","last_undegraded":"2024-09-12T03:04:42.719517+0000","last_fullsized":"2024-09-12T03:04:42.719517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850564+0000","last_change":"2024-09-12T03:03:17.541327+0000","last_active":"2024-09-12T03:04:42.850564+0000","last_peered":"2024-09-12T03:04:42.850564+0000","last_clean":"2024-09-12T03:04:42.850564+0000","last_became_active":"2024-09-12T03:03:17.540959+0000","last_became_peered":"2024-09-12T03:03:17.540959+0000","last_unstale":"2024-09-12T03:04:42.850564+0000","last_undegraded":"2024-09-12T03:04:42.850564+0000","last_fullsized":"2024-09-12T03:04:42.850564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:16.833511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017157+0000","last_change":"2024-09-12T03:03:07.458806+0000","last_active":"2024-09-12T03:04:43.017157+0000","last_peered":"2024-09-12T03:04:43.017157+0000","last_clean":"2024-09-12T03:04:43.017157+0000","last_became_active":"2024-09-12T03:03:07.458501+0000","last_became_peered":"2024-09-12T03:03:07.458501+0000","last_unstale":"2024-09-12T03:04:43.017157+0000","last_undegraded":"2024-09-12T03:04:43.017157+0000","last_fullsized":"2024-09-12T03:04:43.017157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:54.608552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458906+0000","last_change":"2024-09-12T03:03:41.676350+0000","last_active":"2024-09-12T03:04:42.458906+0000","last_peered":"2024-09-12T03:04:42.458906+0000","last_clean":"2024-09-12T03:04:42.458906+0000","last_became_active":"2024-09-12T03:03:41.675935+0000","last_became_peered":"2024-09-12T03:03:41.675935+0000","last_unstale":"2024-09-12T03:04:42.458906+0000","last_undegraded":"2024-09-12T03:04:42.458906+0000","last_fullsized":"2024-09-12T03:04:42.458906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:11.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460124+0000","last_change":"2024-09-12T03:03:17.533670+0000","last_active":"2024-09-12T03:04:42.460124+0000","last_peered":"2024-09-12T03:04:42.460124+0000","last_clean":"2024-09-12T03:04:42.460124+0000","last_became_active":"2024-09-12T03:03:17.530516+0000","last_became_peered":"2024-09-12T03:03:17.530516+0000","last_unstale":"2024-09-12T03:04:42.460124+0000","last_undegraded":"2024-09-12T03:04:42.460124+0000","last_fullsized":"2024-09-12T03:04:42.460124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:02.228503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064753+0000","last_change":"2024-09-12T03:03:07.448760+0000","last_active":"2024-09-12T03:04:43.064753+0000","last_peered":"2024-09-12T03:04:43.064753+0000","last_clean":"2024-09-12T03:04:43.064753+0000","last_became_active":"2024-09-12T03:03:07.447978+0000","last_became_peered":"2024-09-12T03:03:07.447978+0000","last_unstale":"2024-09-12T03:04:43.064753+0000","last_undegraded":"2024-09-12T03:04:43.064753+0000","last_fullsized":"2024-09-12T03:04:43.064753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.319881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:58.901725+0000","last_change":"2024-09-12T03:03:04.427752+0000","last_active":"2024-09-12T03:04:58.901725+0000","last_peered":"2024-09-12T03:04:58.901725+0000","last_clean":"2024-09-12T03:04:58.901725+0000","last_became_active":"2024-09-12T03:03:04.427327+0000","last_became_peered":"2024-09-12T03:03:04.427327+0000","last_unstale":"2024-09-12T03:04:58.901725+0000","last_undegraded":"2024-09-12T03:04:58.901725+0000","last_fullsized":"2024-09-12T03:04:58.901725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:51.271870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457963+0000","last_change":"2024-09-12T03:03:17.520463+0000","last_active":"2024-09-12T03:04:42.457963+0000","last_peered":"2024-09-12T03:04:42.457963+0000","last_clean":"2024-09-12T03:04:42.457963+0000","last_became_active":"2024-09-12T03:03:17.520081+0000","last_became_peered":"2024-09-12T03:03:17.520081+0000","last_unstale":"2024-09-12T03:04:42.457963+0000","last_undegraded":"2024-09-12T03:04:42.457963+0000","last_fullsized":"2024-09-12T03:04:42.457963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:56.707115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016916+0000","last_change":"2024-09-12T03:03:17.532373+0000","last_active":"2024-09-12T03:04:43.016916+0000","last_peered":"2024-09-12T03:04:43.016916+0000","last_clean":"2024-09-12T03:04:43.016916+0000","last_became_active":"2024-09-12T03:03:17.525425+0000","last_became_peered":"2024-09-12T03:03:17.525425+0000","last_unstale":"2024-09-12T03:04:43.016916+0000","last_undegraded":"2024-09-12T03:04:43.016916+0000","last_fullsized":"2024-09-12T03:04:43.016916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:29.811678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460734+0000","last_change":"2024-09-12T03:03:07.445936+0000","last_active":"2024-09-12T03:04:42.460734+0000","last_peered":"2024-09-12T03:04:42.460734+0000","last_clean":"2024-09-12T03:04:42.460734+0000","last_became_active":"2024-09-12T03:03:07.445643+0000","last_became_peered":"2024-09-12T03:03:07.445643+0000","last_unstale":"2024-09-12T03:04:42.460734+0000","last_undegraded":"2024-09-12T03:04:42.460734+0000","last_fullsized":"2024-09-12T03:04:42.460734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:07.477841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718841+0000","last_change":"2024-09-12T03:03:04.424947+0000","last_active":"2024-09-12T03:04:42.718841+0000","last_peered":"2024-09-12T03:04:42.718841+0000","last_clean":"2024-09-12T03:04:42.718841+0000","last_became_active":"2024-09-12T03:03:04.424297+0000","last_became_peered":"2024-09-12T03:03:04.424297+0000","last_unstale":"2024-09-12T03:04:42.718841+0000","last_undegraded":"2024-09-12T03:04:42.718841+0000","last_fullsized":"2024-09-12T03:04:42.718841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:11.187945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464303+0000","last_change":"2024-09-12T03:03:17.530616+0000","last_active":"2024-09-12T03:04:42.464303+0000","last_peered":"2024-09-12T03:04:42.464303+0000","last_clean":"2024-09-12T03:04:42.464303+0000","last_became_active":"2024-09-12T03:03:17.530146+0000","last_became_peered":"2024-09-12T03:03:17.530146+0000","last_unstale":"2024-09-12T03:04:42.464303+0000","last_undegraded":"2024-09-12T03:04:42.464303+0000","last_fullsized":"2024-09-12T03:04:42.464303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:26.892311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457982+0000","last_change":"2024-09-12T03:03:07.448084+0000","last_active":"2024-09-12T03:04:42.457982+0000","last_peered":"2024-09-12T03:04:42.457982+0000","last_clean":"2024-09-12T03:04:42.457982+0000","last_became_active":"2024-09-12T03:03:07.447363+0000","last_became_peered":"2024-09-12T03:03:07.447363+0000","last_unstale":"2024-09-12T03:04:42.457982+0000","last_undegraded":"2024-09-12T03:04:42.457982+0000","last_fullsized":"2024-09-12T03:04:42.457982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:07.451892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884163+0000","last_change":"2024-09-12T03:03:04.430371+0000","last_active":"2024-09-12T03:04:42.884163+0000","last_peered":"2024-09-12T03:04:42.884163+0000","last_clean":"2024-09-12T03:04:42.884163+0000","last_became_active":"2024-09-12T03:03:04.429811+0000","last_became_peered":"2024-09-12T03:03:04.429811+0000","last_unstale":"2024-09-12T03:04:42.884163+0000","last_undegraded":"2024-09-12T03:04:42.884163+0000","last_fullsized":"2024-09-12T03:04:42.884163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:07.443242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542797+0000","last_change":"2024-09-12T03:03:04.422909+0000","last_active":"2024-09-12T03:04:42.542797+0000","last_peered":"2024-09-12T03:04:42.542797+0000","last_clean":"2024-09-12T03:04:42.542797+0000","last_became_active":"2024-09-12T03:03:04.422476+0000","last_became_peered":"2024-09-12T03:03:04.422476+0000","last_unstale":"2024-09-12T03:04:42.542797+0000","last_undegraded":"2024-09-12T03:04:42.542797+0000","last_fullsized":"2024-09-12T03:04:42.542797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719190+0000","last_change":"2024-09-12T03:03:07.454873+0000","last_active":"2024-09-12T03:04:42.719190+0000","last_peered":"2024-09-12T03:04:42.719190+0000","last_clean":"2024-09-12T03:04:42.719190+0000","last_became_active":"2024-09-12T03:03:07.454521+0000","last_became_peered":"2024-09-12T03:03:07.454521+0000","last_unstale":"2024-09-12T03:04:42.719190+0000","last_undegraded":"2024-09-12T03:04:42.719190+0000","last_fullsized":"2024-09-12T03:04:42.719190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:01.710362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459234+0000","last_change":"2024-09-12T03:03:42.456398+0000","last_active":"2024-09-12T03:04:42.459234+0000","last_peered":"2024-09-12T03:04:42.459234+0000","last_clean":"2024-09-12T03:04:42.459234+0000","last_became_active":"2024-09-12T03:03:42.455878+0000","last_became_peered":"2024-09-12T03:03:42.455878+0000","last_unstale":"2024-09-12T03:04:42.459234+0000","last_undegraded":"2024-09-12T03:04:42.459234+0000","last_fullsized":"2024-09-12T03:04:42.459234+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:31.760018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.903504+0000","last_change":"2024-09-12T03:03:04.424039+0000","last_active":"2024-09-12T03:04:59.903504+0000","last_peered":"2024-09-12T03:04:59.903504+0000","last_clean":"2024-09-12T03:04:59.903504+0000","last_became_active":"2024-09-12T03:03:04.422184+0000","last_became_peered":"2024-09-12T03:03:04.422184+0000","last_unstale":"2024-09-12T03:04:59.903504+0000","last_undegraded":"2024-09-12T03:04:59.903504+0000","last_fullsized":"2024-09-12T03:04:59.903504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:07.108262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464950+0000","last_change":"2024-09-12T03:04:42.464950+0000","last_active":"2024-09-12T03:04:42.464950+0000","last_peered":"2024-09-12T03:04:42.464950+0000","last_clean":"2024-09-12T03:04:42.464950+0000","last_became_active":"2024-09-12T03:04:42.464561+0000","last_became_peered":"2024-09-12T03:04:42.464561+0000","last_unstale":"2024-09-12T03:04:42.464950+0000","last_undegraded":"2024-09-12T03:04:42.464950+0000","last_fullsized":"2024-09-12T03:04:42.464950+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:32.665086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459597+0000","last_change":"2024-09-12T03:03:17.521454+0000","last_active":"2024-09-12T03:04:42.459597+0000","last_peered":"2024-09-12T03:04:42.459597+0000","last_clean":"2024-09-12T03:04:42.459597+0000","last_became_active":"2024-09-12T03:03:17.521053+0000","last_became_peered":"2024-09-12T03:03:17.521053+0000","last_unstale":"2024-09-12T03:04:42.459597+0000","last_undegraded":"2024-09-12T03:04:42.459597+0000","last_fullsized":"2024-09-12T03:04:42.459597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:57.088465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463680+0000","last_change":"2024-09-12T03:03:07.451140+0000","last_active":"2024-09-12T03:04:42.463680+0000","last_peered":"2024-09-12T03:04:42.463680+0000","last_clean":"2024-09-12T03:04:42.463680+0000","last_became_active":"2024-09-12T03:03:07.450775+0000","last_became_peered":"2024-09-12T03:03:07.450775+0000","last_unstale":"2024-09-12T03:04:42.463680+0000","last_undegraded":"2024-09-12T03:04:42.463680+0000","last_fullsized":"2024-09-12T03:04:42.463680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:12.131080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459507+0000","last_change":"2024-09-12T03:03:07.445602+0000","last_active":"2024-09-12T03:04:42.459507+0000","last_peered":"2024-09-12T03:04:42.459507+0000","last_clean":"2024-09-12T03:04:42.459507+0000","last_became_active":"2024-09-12T03:03:07.445210+0000","last_became_peered":"2024-09-12T03:03:07.445210+0000","last_unstale":"2024-09-12T03:04:42.459507+0000","last_undegraded":"2024-09-12T03:04:42.459507+0000","last_fullsized":"2024-09-12T03:04:42.459507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:48.044322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459097+0000","last_change":"2024-09-12T03:03:07.453006+0000","last_active":"2024-09-12T03:04:42.459097+0000","last_peered":"2024-09-12T03:04:42.459097+0000","last_clean":"2024-09-12T03:04:42.459097+0000","last_became_active":"2024-09-12T03:03:07.452608+0000","last_became_peered":"2024-09-12T03:03:07.452608+0000","last_unstale":"2024-09-12T03:04:42.459097+0000","last_undegraded":"2024-09-12T03:04:42.459097+0000","last_fullsized":"2024-09-12T03:04:42.459097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:22.443868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016781+0000","last_change":"2024-09-12T03:03:07.457284+0000","last_active":"2024-09-12T03:04:43.016781+0000","last_peered":"2024-09-12T03:04:43.016781+0000","last_clean":"2024-09-12T03:04:43.016781+0000","last_became_active":"2024-09-12T03:03:07.456964+0000","last_became_peered":"2024-09-12T03:03:07.456964+0000","last_unstale":"2024-09-12T03:04:43.016781+0000","last_undegraded":"2024-09-12T03:04:43.016781+0000","last_fullsized":"2024-09-12T03:04:43.016781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.525035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884816+0000","last_change":"2024-09-12T03:03:07.451622+0000","last_active":"2024-09-12T03:04:42.884816+0000","last_peered":"2024-09-12T03:04:42.884816+0000","last_clean":"2024-09-12T03:04:42.884816+0000","last_became_active":"2024-09-12T03:03:07.451240+0000","last_became_peered":"2024-09-12T03:03:07.451240+0000","last_unstale":"2024-09-12T03:04:42.884816+0000","last_undegraded":"2024-09-12T03:04:42.884816+0000","last_fullsized":"2024-09-12T03:04:42.884816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:17.357741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.465641+0000","last_change":"2024-09-12T03:04:42.465641+0000","last_active":"2024-09-12T03:04:42.465641+0000","last_peered":"2024-09-12T03:04:42.465641+0000","last_clean":"2024-09-12T03:04:42.465641+0000","last_became_active":"2024-09-12T03:04:42.465303+0000","last_became_peered":"2024-09-12T03:04:42.465303+0000","last_unstale":"2024-09-12T03:04:42.465641+0000","last_undegraded":"2024-09-12T03:04:42.465641+0000","last_fullsized":"2024-09-12T03:04:42.465641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:31.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459670+0000","last_change":"2024-09-12T03:03:07.444125+0000","last_active":"2024-09-12T03:04:42.459670+0000","last_peered":"2024-09-12T03:04:42.459670+0000","last_clean":"2024-09-12T03:04:42.459670+0000","last_became_active":"2024-09-12T03:03:07.443799+0000","last_became_peered":"2024-09-12T03:03:07.443799+0000","last_unstale":"2024-09-12T03:04:42.459670+0000","last_undegraded":"2024-09-12T03:04:42.459670+0000","last_fullsized":"2024-09-12T03:04:42.459670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:49.573684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459566+0000","last_change":"2024-09-12T03:03:07.446562+0000","last_active":"2024-09-12T03:04:42.459566+0000","last_peered":"2024-09-12T03:04:42.459566+0000","last_clean":"2024-09-12T03:04:42.459566+0000","last_became_active":"2024-09-12T03:03:07.445539+0000","last_became_peered":"2024-09-12T03:03:07.445539+0000","last_unstale":"2024-09-12T03:04:42.459566+0000","last_undegraded":"2024-09-12T03:04:42.459566+0000","last_fullsized":"2024-09-12T03:04:42.459566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:06.939290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718642+0000","last_change":"2024-09-12T03:03:07.444899+0000","last_active":"2024-09-12T03:04:42.718642+0000","last_peered":"2024-09-12T03:04:42.718642+0000","last_clean":"2024-09-12T03:04:42.718642+0000","last_became_active":"2024-09-12T03:03:07.443309+0000","last_became_peered":"2024-09-12T03:03:07.443309+0000","last_unstale":"2024-09-12T03:04:42.718642+0000","last_undegraded":"2024-09-12T03:04:42.718642+0000","last_fullsized":"2024-09-12T03:04:42.718642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:06.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.466478+0000","last_change":"2024-09-12T03:04:42.465614+0000","last_active":"2024-09-12T03:04:42.466478+0000","last_peered":"2024-09-12T03:04:42.466478+0000","last_clean":"2024-09-12T03:04:42.466478+0000","last_became_active":"2024-09-12T03:04:42.465250+0000","last_became_peered":"2024-09-12T03:04:42.465250+0000","last_unstale":"2024-09-12T03:04:42.466478+0000","last_undegraded":"2024-09-12T03:04:42.466478+0000","last_fullsized":"2024-09-12T03:04:42.466478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:29.305821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064550+0000","last_change":"2024-09-12T03:03:07.445924+0000","last_active":"2024-09-12T03:04:43.064550+0000","last_peered":"2024-09-12T03:04:43.064550+0000","last_clean":"2024-09-12T03:04:43.064550+0000","last_became_active":"2024-09-12T03:03:07.445449+0000","last_became_peered":"2024-09-12T03:03:07.445449+0000","last_unstale":"2024-09-12T03:04:43.064550+0000","last_undegraded":"2024-09-12T03:04:43.064550+0000","last_fullsized":"2024-09-12T03:04:43.064550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:42.040044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884569+0000","last_change":"2024-09-12T03:03:07.449653+0000","last_active":"2024-09-12T03:04:42.884569+0000","last_peered":"2024-09-12T03:04:42.884569+0000","last_clean":"2024-09-12T03:04:42.884569+0000","last_became_active":"2024-09-12T03:03:07.445691+0000","last_became_peered":"2024-09-12T03:03:07.445691+0000","last_unstale":"2024-09-12T03:04:42.884569+0000","last_undegraded":"2024-09-12T03:04:42.884569+0000","last_fullsized":"2024-09-12T03:04:42.884569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:53.210238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.465695+0000","last_change":"2024-09-12T03:04:42.465695+0000","last_active":"2024-09-12T03:04:42.465695+0000","last_peered":"2024-09-12T03:04:42.465695+0000","last_clean":"2024-09-12T03:04:42.465695+0000","last_became_active":"2024-09-12T03:04:42.465338+0000","last_became_peered":"2024-09-12T03:04:42.465338+0000","last_unstale":"2024-09-12T03:04:42.465695+0000","last_undegraded":"2024-09-12T03:04:42.465695+0000","last_fullsized":"2024-09-12T03:04:42.465695+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:22.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459611+0000","last_change":"2024-09-12T03:03:07.441645+0000","last_active":"2024-09-12T03:04:42.459611+0000","last_peered":"2024-09-12T03:04:42.459611+0000","last_clean":"2024-09-12T03:04:42.459611+0000","last_became_active":"2024-09-12T03:03:07.441191+0000","last_became_peered":"2024-09-12T03:03:07.441191+0000","last_unstale":"2024-09-12T03:04:42.459611+0000","last_undegraded":"2024-09-12T03:04:42.459611+0000","last_fullsized":"2024-09-12T03:04:42.459611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:40.873214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460768+0000","last_change":"2024-09-12T03:03:07.446707+0000","last_active":"2024-09-12T03:04:42.460768+0000","last_peered":"2024-09-12T03:04:42.460768+0000","last_clean":"2024-09-12T03:04:42.460768+0000","last_became_active":"2024-09-12T03:03:07.446301+0000","last_became_peered":"2024-09-12T03:03:07.446301+0000","last_unstale":"2024-09-12T03:04:42.460768+0000","last_undegraded":"2024-09-12T03:04:42.460768+0000","last_fullsized":"2024-09-12T03:04:42.460768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:30.610892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464637+0000","last_change":"2024-09-12T03:03:07.449478+0000","last_active":"2024-09-12T03:04:42.464637+0000","last_peered":"2024-09-12T03:04:42.464637+0000","last_clean":"2024-09-12T03:04:42.464637+0000","last_became_active":"2024-09-12T03:03:07.448830+0000","last_became_peered":"2024-09-12T03:03:07.448830+0000","last_unstale":"2024-09-12T03:04:42.464637+0000","last_undegraded":"2024-09-12T03:04:42.464637+0000","last_fullsized":"2024-09-12T03:04:42.464637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:55.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064337+0000","last_change":"2024-09-12T03:03:07.449866+0000","last_active":"2024-09-12T03:04:43.064337+0000","last_peered":"2024-09-12T03:04:43.064337+0000","last_clean":"2024-09-12T03:04:43.064337+0000","last_became_active":"2024-09-12T03:03:07.448934+0000","last_became_peered":"2024-09-12T03:03:07.448934+0000","last_unstale":"2024-09-12T03:04:43.064337+0000","last_undegraded":"2024-09-12T03:04:43.064337+0000","last_fullsized":"2024-09-12T03:04:43.064337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:24.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016515+0000","last_change":"2024-09-12T03:03:41.673404+0000","last_active":"2024-09-12T03:04:43.016515+0000","last_peered":"2024-09-12T03:04:43.016515+0000","last_clean":"2024-09-12T03:04:43.016515+0000","last_became_active":"2024-09-12T03:03:41.673022+0000","last_became_peered":"2024-09-12T03:03:41.673022+0000","last_unstale":"2024-09-12T03:04:43.016515+0000","last_undegraded":"2024-09-12T03:04:43.016515+0000","last_fullsized":"2024-09-12T03:04:43.016515+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:44.083090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016583+0000","last_change":"2024-09-12T03:03:07.455058+0000","last_active":"2024-09-12T03:04:43.016583+0000","last_peered":"2024-09-12T03:04:43.016583+0000","last_clean":"2024-09-12T03:04:43.016583+0000","last_became_active":"2024-09-12T03:03:07.449331+0000","last_became_peered":"2024-09-12T03:03:07.449331+0000","last_unstale":"2024-09-12T03:04:43.016583+0000","last_undegraded":"2024-09-12T03:04:43.016583+0000","last_fullsized":"2024-09-12T03:04:43.016583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:30.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884540+0000","last_change":"2024-09-12T03:03:07.447193+0000","last_active":"2024-09-12T03:04:42.884540+0000","last_peered":"2024-09-12T03:04:42.884540+0000","last_clean":"2024-09-12T03:04:42.884540+0000","last_became_active":"2024-09-12T03:03:07.445206+0000","last_became_peered":"2024-09-12T03:03:07.445206+0000","last_unstale":"2024-09-12T03:04:42.884540+0000","last_undegraded":"2024-09-12T03:04:42.884540+0000","last_fullsized":"2024-09-12T03:04:42.884540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:28.919293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.851095+0000","last_change":"2024-09-12T03:03:07.442621+0000","last_active":"2024-09-12T03:04:42.851095+0000","last_peered":"2024-09-12T03:04:42.851095+0000","last_clean":"2024-09-12T03:04:42.851095+0000","last_became_active":"2024-09-12T03:03:07.441124+0000","last_became_peered":"2024-09-12T03:03:07.441124+0000","last_unstale":"2024-09-12T03:04:42.851095+0000","last_undegraded":"2024-09-12T03:04:42.851095+0000","last_fullsized":"2024-09-12T03:04:42.851095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:08.442627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458916+0000","last_change":"2024-09-12T03:03:07.443619+0000","last_active":"2024-09-12T03:04:42.458916+0000","last_peered":"2024-09-12T03:04:42.458916+0000","last_clean":"2024-09-12T03:04:42.458916+0000","last_became_active":"2024-09-12T03:03:07.443290+0000","last_became_peered":"2024-09-12T03:03:07.443290+0000","last_unstale":"2024-09-12T03:04:42.458916+0000","last_undegraded":"2024-09-12T03:04:42.458916+0000","last_fullsized":"2024-09-12T03:04:42.458916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:43.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.543277+0000","last_change":"2024-09-12T03:03:07.454164+0000","last_active":"2024-09-12T03:04:42.543277+0000","last_peered":"2024-09-12T03:04:42.543277+0000","last_clean":"2024-09-12T03:04:42.543277+0000","last_became_active":"2024-09-12T03:03:07.453835+0000","last_became_peered":"2024-09-12T03:03:07.453835+0000","last_unstale":"2024-09-12T03:04:42.543277+0000","last_undegraded":"2024-09-12T03:04:42.543277+0000","last_fullsized":"2024-09-12T03:04:42.543277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:00.494720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064951+0000","last_change":"2024-09-12T03:03:07.446854+0000","last_active":"2024-09-12T03:04:43.064951+0000","last_peered":"2024-09-12T03:04:43.064951+0000","last_clean":"2024-09-12T03:04:43.064951+0000","last_became_active":"2024-09-12T03:03:07.446411+0000","last_became_peered":"2024-09-12T03:03:07.446411+0000","last_unstale":"2024-09-12T03:04:43.064951+0000","last_undegraded":"2024-09-12T03:04:43.064951+0000","last_fullsized":"2024-09-12T03:04:43.064951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:54.917686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457958+0000","last_change":"2024-09-12T03:03:07.448332+0000","last_active":"2024-09-12T03:04:42.457958+0000","last_peered":"2024-09-12T03:04:42.457958+0000","last_clean":"2024-09-12T03:04:42.457958+0000","last_became_active":"2024-09-12T03:03:07.447776+0000","last_became_peered":"2024-09-12T03:03:07.447776+0000","last_unstale":"2024-09-12T03:04:42.457958+0000","last_undegraded":"2024-09-12T03:04:42.457958+0000","last_fullsized":"2024-09-12T03:04:42.457958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:08.531088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458250+0000","last_change":"2024-09-12T03:03:07.449781+0000","last_active":"2024-09-12T03:04:42.458250+0000","last_peered":"2024-09-12T03:04:42.458250+0000","last_clean":"2024-09-12T03:04:42.458250+0000","last_became_active":"2024-09-12T03:03:07.449359+0000","last_became_peered":"2024-09-12T03:03:07.449359+0000","last_unstale":"2024-09-12T03:04:42.458250+0000","last_undegraded":"2024-09-12T03:04:42.458250+0000","last_fullsized":"2024-09-12T03:04:42.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:11.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542793+0000","last_change":"2024-09-12T03:03:07.453881+0000","last_active":"2024-09-12T03:04:42.542793+0000","last_peered":"2024-09-12T03:04:42.542793+0000","last_clean":"2024-09-12T03:04:42.542793+0000","last_became_active":"2024-09-12T03:03:07.453455+0000","last_became_peered":"2024-09-12T03:03:07.453455+0000","last_unstale":"2024-09-12T03:04:42.542793+0000","last_undegraded":"2024-09-12T03:04:42.542793+0000","last_fullsized":"2024-09-12T03:04:42.542793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:18:46.797598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462235+0000","last_change":"2024-09-12T03:03:07.459332+0000","last_active":"2024-09-12T03:04:42.462235+0000","last_peered":"2024-09-12T03:04:42.462235+0000","last_clean":"2024-09-12T03:04:42.462235+0000","last_became_active":"2024-09-12T03:03:07.459032+0000","last_became_peered":"2024-09-12T03:03:07.459032+0000","last_unstale":"2024-09-12T03:04:42.462235+0000","last_undegraded":"2024-09-12T03:04:42.462235+0000","last_fullsized":"2024-09-12T03:04:42.462235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:16.702782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.851092+0000","last_change":"2024-09-12T03:03:07.441696+0000","last_active":"2024-09-12T03:04:42.851092+0000","last_peered":"2024-09-12T03:04:42.851092+0000","last_clean":"2024-09-12T03:04:42.851092+0000","last_became_active":"2024-09-12T03:03:07.440575+0000","last_became_peered":"2024-09-12T03:03:07.440575+0000","last_unstale":"2024-09-12T03:04:42.851092+0000","last_undegraded":"2024-09-12T03:04:42.851092+0000","last_fullsized":"2024-09-12T03:04:42.851092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:26.049893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016765+0000","last_change":"2024-09-12T03:03:07.454945+0000","last_active":"2024-09-12T03:04:43.016765+0000","last_peered":"2024-09-12T03:04:43.016765+0000","last_clean":"2024-09-12T03:04:43.016765+0000","last_became_active":"2024-09-12T03:03:07.452368+0000","last_became_peered":"2024-09-12T03:03:07.452368+0000","last_unstale":"2024-09-12T03:04:43.016765+0000","last_undegraded":"2024-09-12T03:04:43.016765+0000","last_fullsized":"2024-09-12T03:04:43.016765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:21.169859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016920+0000","last_change":"2024-09-12T03:03:07.454230+0000","last_active":"2024-09-12T03:04:43.016920+0000","last_peered":"2024-09-12T03:04:43.016920+0000","last_clean":"2024-09-12T03:04:43.016920+0000","last_became_active":"2024-09-12T03:03:07.453913+0000","last_became_peered":"2024-09-12T03:03:07.453913+0000","last_unstale":"2024-09-12T03:04:43.016920+0000","last_undegraded":"2024-09-12T03:04:43.016920+0000","last_fullsized":"2024-09-12T03:04:43.016920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:56.264048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464571+0000","last_change":"2024-09-12T03:03:07.449227+0000","last_active":"2024-09-12T03:04:42.464571+0000","last_peered":"2024-09-12T03:04:42.464571+0000","last_clean":"2024-09-12T03:04:42.464571+0000","last_became_active":"2024-09-12T03:03:07.448605+0000","last_became_peered":"2024-09-12T03:03:07.448605+0000","last_unstale":"2024-09-12T03:04:42.464571+0000","last_undegraded":"2024-09-12T03:04:42.464571+0000","last_fullsized":"2024-09-12T03:04:42.464571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:06.420752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":409,"num_read_kb":277,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481829,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":554372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712208,"statfs":{"total":95995035648,"available":95961300992,"internally_reserved":0,"allocated":1798144,"data_stored":1204301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":558704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1122304,"data_stored":554528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1716224,"data_stored":1150439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":549784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1724416,"data_stored":1146027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":554434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":549978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1196032,"data_stored":614098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1171456,"data_stored":612355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1126400,"data_stored":551172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:05:03.227 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:05:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:03.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:03 smithi057 ceph-mon[30361]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:03 smithi057 ceph-mon[30361]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:03 smithi204 ceph-mon[35013]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:03 smithi204 ceph-mon[35013]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:03 smithi181 ceph-mon[35320]: pgmap v367: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:03 smithi181 ceph-mon[35320]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:04.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 -- 172.21.15.57:0/365985286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 msgr2=0x7ff29c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 --2- 172.21.15.57:0/365985286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c1089f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff284009900 tx=0x7ff28402a410 comp rx=0 tx=0).stop 2024-09-12T03:05:04.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 -- 172.21.15.57:0/365985286 shutdown_connections 2024-09-12T03:05:04.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 --2- 172.21.15.57:0/365985286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 --2- 172.21.15.57:0/365985286 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c1089f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 --2- 172.21.15.57:0/365985286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 0x7ff29c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 -- 172.21.15.57:0/365985286 >> 172.21.15.57:0/365985286 conn(0x7ff29c0ffa90 msgr2=0x7ff29c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 -- 172.21.15.57:0/365985286 shutdown_connections 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.175+0000 7ff2a2200640 1 -- 172.21.15.57:0/365985286 wait complete. 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.176+0000 7ff2a2200640 1 Processor -- start 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.176+0000 7ff2a2200640 1 -- start start 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 0x7ff29c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c114190 con 0x7ff29c1049a0 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c114010 con 0x7ff29c103ff0 2024-09-12T03:05:04.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff2a2200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c114310 con 0x7ff29c109120 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff292a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff292a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42830/0 (socket says 172.21.15.57:42830) 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff292a00640 1 -- 172.21.15.57:0/3082628470 learned_addr learned my addr 172.21.15.57:0/3082628470 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff29b400640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 0x7ff29c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff29be00640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff29be00640 1 -- 172.21.15.57:0/3082628470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 msgr2=0x7ff29c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff29be00640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 0x7ff29c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.177+0000 7ff29be00640 1 -- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 msgr2=0x7ff29c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff29be00640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff29be00640 1 -- 172.21.15.57:0/3082628470 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c19f5a0 con 0x7ff29c109120 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff29be00640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c19eec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff29400e970 tx=0x7ff29400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29400ccc0 con 0x7ff29c109120 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c19f890 con 0x7ff29c109120 2024-09-12T03:05:04.199 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff294005020 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294004a00 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.178+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c1a7170 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.179+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c1043d0 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.181+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff2940051d0 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.183+0000 7ff298c00640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 0x7ff2800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.183+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7ff2940cf590 con 0x7ff29c109120 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.183+0000 7ff29b400640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 0x7ff2800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.183+0000 7ff29b400640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 0x7ff2800a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.185+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff29409bff0 con 0x7ff29c109120 2024-09-12T03:05:04.368 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.368+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff29c100da0 con 0x7ff2800a2c10 2024-09-12T03:05:04.376 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.376+0000 7ff298c00640 1 -- 172.21.15.57:0/3082628470 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340361 (secure 0 0 0) 0x7ff29c100da0 con 0x7ff2800a2c10 2024-09-12T03:05:04.380 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:04.382 INFO:teuthology.orchestra.run.smithi057.stderr:dumped all 2024-09-12T03:05:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 msgr2=0x7ff2800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 0x7ff2800a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff28c005fd0 tx=0x7ff28c009760 comp rx=0 tx=0).stop 2024-09-12T03:05:04.384 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 msgr2=0x7ff29c19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c19eec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff29400e970 tx=0x7ff29400ee30 comp rx=0 tx=0).stop 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 shutdown_connections 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7ff2800a2c10 0x7ff2800a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff29c109120 0x7ff29c19eec0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff29c1049a0 0x7ff29c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 --2- 172.21.15.57:0/3082628470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff29c103ff0 0x7ff29c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 >> 172.21.15.57:0/3082628470 conn(0x7ff29c0ffa90 msgr2=0x7ff29c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 shutdown_connections 2024-09-12T03:05:04.385 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:04.381+0000 7ff2a2200640 1 -- 172.21.15.57:0/3082628470 wait complete. 2024-09-12T03:05:05.208 INFO:teuthology.orchestra.run.smithi057.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-12T03:05:03.741204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657266,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":550,"num_read_kb":396,"num_write":350,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372688,"kb_used_data":15056,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569136,"statfs":{"total":1151940427776,"available":1151558795264,"internally_reserved":0,"allocated":15417344,"data_stored":8600192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003787"},"pg_stats":[{"pgid":"2.1f","version":"94'9","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461083+0000","last_change":"2024-09-12T03:03:04.425029+0000","last_active":"2024-09-12T03:04:42.461083+0000","last_peered":"2024-09-12T03:04:42.461083+0000","last_clean":"2024-09-12T03:04:42.461083+0000","last_became_active":"2024-09-12T03:03:04.423326+0000","last_became_peered":"2024-09-12T03:03:04.423326+0000","last_unstale":"2024-09-12T03:04:42.461083+0000","last_undegraded":"2024-09-12T03:04:42.461083+0000","last_fullsized":"2024-09-12T03:04:42.461083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:26:28.269107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459552+0000","last_change":"2024-09-12T03:03:07.449348+0000","last_active":"2024-09-12T03:04:42.459552+0000","last_peered":"2024-09-12T03:04:42.459552+0000","last_clean":"2024-09-12T03:04:42.459552+0000","last_became_active":"2024-09-12T03:03:07.448813+0000","last_became_peered":"2024-09-12T03:03:07.448813+0000","last_unstale":"2024-09-12T03:04:42.459552+0000","last_undegraded":"2024-09-12T03:04:42.459552+0000","last_fullsized":"2024-09-12T03:04:42.459552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:26:31.404583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459020+0000","last_change":"2024-09-12T03:03:17.530646+0000","last_active":"2024-09-12T03:04:42.459020+0000","last_peered":"2024-09-12T03:04:42.459020+0000","last_clean":"2024-09-12T03:04:42.459020+0000","last_became_active":"2024-09-12T03:03:17.530228+0000","last_became_peered":"2024-09-12T03:03:17.530228+0000","last_unstale":"2024-09-12T03:04:42.459020+0000","last_undegraded":"2024-09-12T03:04:42.459020+0000","last_fullsized":"2024-09-12T03:04:42.459020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:09.763496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459795+0000","last_change":"2024-09-12T03:03:04.415539+0000","last_active":"2024-09-12T03:04:42.459795+0000","last_peered":"2024-09-12T03:04:42.459795+0000","last_clean":"2024-09-12T03:04:42.459795+0000","last_became_active":"2024-09-12T03:03:04.415163+0000","last_became_peered":"2024-09-12T03:03:04.415163+0000","last_unstale":"2024-09-12T03:04:42.459795+0000","last_undegraded":"2024-09-12T03:04:42.459795+0000","last_fullsized":"2024-09-12T03:04:42.459795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:12.628687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017157+0000","last_change":"2024-09-12T03:03:07.447808+0000","last_active":"2024-09-12T03:04:43.017157+0000","last_peered":"2024-09-12T03:04:43.017157+0000","last_clean":"2024-09-12T03:04:43.017157+0000","last_became_active":"2024-09-12T03:03:07.446760+0000","last_became_peered":"2024-09-12T03:03:07.446760+0000","last_unstale":"2024-09-12T03:04:43.017157+0000","last_undegraded":"2024-09-12T03:04:43.017157+0000","last_fullsized":"2024-09-12T03:04:43.017157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:56.758115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850298+0000","last_change":"2024-09-12T03:03:17.530408+0000","last_active":"2024-09-12T03:04:42.850298+0000","last_peered":"2024-09-12T03:04:42.850298+0000","last_clean":"2024-09-12T03:04:42.850298+0000","last_became_active":"2024-09-12T03:03:17.530073+0000","last_became_peered":"2024-09-12T03:03:17.530073+0000","last_unstale":"2024-09-12T03:04:42.850298+0000","last_undegraded":"2024-09-12T03:04:42.850298+0000","last_fullsized":"2024-09-12T03:04:42.850298+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:57.199786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462801+0000","last_change":"2024-09-12T03:03:04.422168+0000","last_active":"2024-09-12T03:04:42.462801+0000","last_peered":"2024-09-12T03:04:42.462801+0000","last_clean":"2024-09-12T03:04:42.462801+0000","last_became_active":"2024-09-12T03:03:04.421739+0000","last_became_peered":"2024-09-12T03:03:04.421739+0000","last_unstale":"2024-09-12T03:04:42.462801+0000","last_undegraded":"2024-09-12T03:04:42.462801+0000","last_fullsized":"2024-09-12T03:04:42.462801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:28.080796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064640+0000","last_change":"2024-09-12T03:03:07.445256+0000","last_active":"2024-09-12T03:04:43.064640+0000","last_peered":"2024-09-12T03:04:43.064640+0000","last_clean":"2024-09-12T03:04:43.064640+0000","last_became_active":"2024-09-12T03:03:07.444919+0000","last_became_peered":"2024-09-12T03:03:07.444919+0000","last_unstale":"2024-09-12T03:04:43.064640+0000","last_undegraded":"2024-09-12T03:04:43.064640+0000","last_fullsized":"2024-09-12T03:04:43.064640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:57:42.826440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850739+0000","last_change":"2024-09-12T03:03:42.456763+0000","last_active":"2024-09-12T03:04:42.850739+0000","last_peered":"2024-09-12T03:04:42.850739+0000","last_clean":"2024-09-12T03:04:42.850739+0000","last_became_active":"2024-09-12T03:03:42.456267+0000","last_became_peered":"2024-09-12T03:03:42.456267+0000","last_unstale":"2024-09-12T03:04:42.850739+0000","last_undegraded":"2024-09-12T03:04:42.850739+0000","last_fullsized":"2024-09-12T03:04:42.850739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:17.092878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462686+0000","last_change":"2024-09-12T03:03:04.423817+0000","last_active":"2024-09-12T03:04:42.462686+0000","last_peered":"2024-09-12T03:04:42.462686+0000","last_clean":"2024-09-12T03:04:42.462686+0000","last_became_active":"2024-09-12T03:03:04.422617+0000","last_became_peered":"2024-09-12T03:03:04.422617+0000","last_unstale":"2024-09-12T03:04:42.462686+0000","last_undegraded":"2024-09-12T03:04:42.462686+0000","last_fullsized":"2024-09-12T03:04:42.462686+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:33.309338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719320+0000","last_change":"2024-09-12T03:03:07.454768+0000","last_active":"2024-09-12T03:04:42.719320+0000","last_peered":"2024-09-12T03:04:42.719320+0000","last_clean":"2024-09-12T03:04:42.719320+0000","last_became_active":"2024-09-12T03:03:07.454414+0000","last_became_peered":"2024-09-12T03:03:07.454414+0000","last_unstale":"2024-09-12T03:04:42.719320+0000","last_undegraded":"2024-09-12T03:04:42.719320+0000","last_fullsized":"2024-09-12T03:04:42.719320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:39.968938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850377+0000","last_change":"2024-09-12T03:03:17.524984+0000","last_active":"2024-09-12T03:04:42.850377+0000","last_peered":"2024-09-12T03:04:42.850377+0000","last_clean":"2024-09-12T03:04:42.850377+0000","last_became_active":"2024-09-12T03:03:17.524516+0000","last_became_peered":"2024-09-12T03:03:17.524516+0000","last_unstale":"2024-09-12T03:04:42.850377+0000","last_undegraded":"2024-09-12T03:04:42.850377+0000","last_fullsized":"2024-09-12T03:04:42.850377+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:23:34.342631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458382+0000","last_change":"2024-09-12T03:03:07.450348+0000","last_active":"2024-09-12T03:04:42.458382+0000","last_peered":"2024-09-12T03:04:42.458382+0000","last_clean":"2024-09-12T03:04:42.458382+0000","last_became_active":"2024-09-12T03:03:07.448968+0000","last_became_peered":"2024-09-12T03:03:07.448968+0000","last_unstale":"2024-09-12T03:04:42.458382+0000","last_undegraded":"2024-09-12T03:04:42.458382+0000","last_fullsized":"2024-09-12T03:04:42.458382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:19.053409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463637+0000","last_change":"2024-09-12T03:03:41.676526+0000","last_active":"2024-09-12T03:04:42.463637+0000","last_peered":"2024-09-12T03:04:42.463637+0000","last_clean":"2024-09-12T03:04:42.463637+0000","last_became_active":"2024-09-12T03:03:41.673301+0000","last_became_peered":"2024-09-12T03:03:41.673301+0000","last_unstale":"2024-09-12T03:04:42.463637+0000","last_undegraded":"2024-09-12T03:04:42.463637+0000","last_fullsized":"2024-09-12T03:04:42.463637+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:06:04.648668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850619+0000","last_change":"2024-09-12T03:03:17.526846+0000","last_active":"2024-09-12T03:04:42.850619+0000","last_peered":"2024-09-12T03:04:42.850619+0000","last_clean":"2024-09-12T03:04:42.850619+0000","last_became_active":"2024-09-12T03:03:17.526412+0000","last_became_peered":"2024-09-12T03:03:17.526412+0000","last_unstale":"2024-09-12T03:04:42.850619+0000","last_undegraded":"2024-09-12T03:04:42.850619+0000","last_fullsized":"2024-09-12T03:04:42.850619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:16:59.381000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458566+0000","last_change":"2024-09-12T03:03:07.453989+0000","last_active":"2024-09-12T03:04:42.458566+0000","last_peered":"2024-09-12T03:04:42.458566+0000","last_clean":"2024-09-12T03:04:42.458566+0000","last_became_active":"2024-09-12T03:03:07.451796+0000","last_became_peered":"2024-09-12T03:03:07.451796+0000","last_unstale":"2024-09-12T03:04:42.458566+0000","last_undegraded":"2024-09-12T03:04:42.458566+0000","last_fullsized":"2024-09-12T03:04:42.458566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:27:27.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459110+0000","last_change":"2024-09-12T03:03:04.423756+0000","last_active":"2024-09-12T03:04:42.459110+0000","last_peered":"2024-09-12T03:04:42.459110+0000","last_clean":"2024-09-12T03:04:42.459110+0000","last_became_active":"2024-09-12T03:03:04.423075+0000","last_became_peered":"2024-09-12T03:03:04.423075+0000","last_unstale":"2024-09-12T03:04:42.459110+0000","last_undegraded":"2024-09-12T03:04:42.459110+0000","last_fullsized":"2024-09-12T03:04:42.459110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:44.201739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850984+0000","last_change":"2024-09-12T03:03:17.532906+0000","last_active":"2024-09-12T03:04:42.850984+0000","last_peered":"2024-09-12T03:04:42.850984+0000","last_clean":"2024-09-12T03:04:42.850984+0000","last_became_active":"2024-09-12T03:03:17.532343+0000","last_became_peered":"2024-09-12T03:03:17.532343+0000","last_unstale":"2024-09-12T03:04:42.850984+0000","last_undegraded":"2024-09-12T03:04:42.850984+0000","last_fullsized":"2024-09-12T03:04:42.850984+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:20:05.888756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463613+0000","last_change":"2024-09-12T03:03:07.450375+0000","last_active":"2024-09-12T03:04:42.463613+0000","last_peered":"2024-09-12T03:04:42.463613+0000","last_clean":"2024-09-12T03:04:42.463613+0000","last_became_active":"2024-09-12T03:03:07.450061+0000","last_became_peered":"2024-09-12T03:03:07.450061+0000","last_unstale":"2024-09-12T03:04:42.463613+0000","last_undegraded":"2024-09-12T03:04:42.463613+0000","last_fullsized":"2024-09-12T03:04:42.463613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:54:05.922237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458364+0000","last_change":"2024-09-12T03:03:04.423664+0000","last_active":"2024-09-12T03:04:42.458364+0000","last_peered":"2024-09-12T03:04:42.458364+0000","last_clean":"2024-09-12T03:04:42.458364+0000","last_became_active":"2024-09-12T03:03:04.423285+0000","last_became_peered":"2024-09-12T03:03:04.423285+0000","last_unstale":"2024-09-12T03:04:42.458364+0000","last_undegraded":"2024-09-12T03:04:42.458364+0000","last_fullsized":"2024-09-12T03:04:42.458364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:57.782484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460713+0000","last_change":"2024-09-12T03:03:17.535385+0000","last_active":"2024-09-12T03:04:42.460713+0000","last_peered":"2024-09-12T03:04:42.460713+0000","last_clean":"2024-09-12T03:04:42.460713+0000","last_became_active":"2024-09-12T03:03:17.534775+0000","last_became_peered":"2024-09-12T03:03:17.534775+0000","last_unstale":"2024-09-12T03:04:42.460713+0000","last_undegraded":"2024-09-12T03:04:42.460713+0000","last_fullsized":"2024-09-12T03:04:42.460713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:02.549462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462139+0000","last_change":"2024-09-12T03:03:07.442786+0000","last_active":"2024-09-12T03:04:42.462139+0000","last_peered":"2024-09-12T03:04:42.462139+0000","last_clean":"2024-09-12T03:04:42.462139+0000","last_became_active":"2024-09-12T03:03:07.442456+0000","last_became_peered":"2024-09-12T03:03:07.442456+0000","last_unstale":"2024-09-12T03:04:42.462139+0000","last_undegraded":"2024-09-12T03:04:42.462139+0000","last_fullsized":"2024-09-12T03:04:42.462139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:27.882361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064499+0000","last_change":"2024-09-12T03:03:41.675913+0000","last_active":"2024-09-12T03:04:43.064499+0000","last_peered":"2024-09-12T03:04:43.064499+0000","last_clean":"2024-09-12T03:04:43.064499+0000","last_became_active":"2024-09-12T03:03:41.675434+0000","last_became_peered":"2024-09-12T03:03:41.675434+0000","last_unstale":"2024-09-12T03:04:43.064499+0000","last_undegraded":"2024-09-12T03:04:43.064499+0000","last_fullsized":"2024-09-12T03:04:43.064499+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:36.484404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850833+0000","last_change":"2024-09-12T03:03:42.177089+0000","last_active":"2024-09-12T03:04:42.850833+0000","last_peered":"2024-09-12T03:04:42.850833+0000","last_clean":"2024-09-12T03:04:42.850833+0000","last_became_active":"2024-09-12T03:03:42.176602+0000","last_became_peered":"2024-09-12T03:03:42.176602+0000","last_unstale":"2024-09-12T03:04:42.850833+0000","last_undegraded":"2024-09-12T03:04:42.850833+0000","last_fullsized":"2024-09-12T03:04:42.850833+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:04.336826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017043+0000","last_change":"2024-09-12T03:03:07.457972+0000","last_active":"2024-09-12T03:04:43.017043+0000","last_peered":"2024-09-12T03:04:43.017043+0000","last_clean":"2024-09-12T03:04:43.017043+0000","last_became_active":"2024-09-12T03:03:07.457681+0000","last_became_peered":"2024-09-12T03:03:07.457681+0000","last_unstale":"2024-09-12T03:04:43.017043+0000","last_undegraded":"2024-09-12T03:04:43.017043+0000","last_fullsized":"2024-09-12T03:04:43.017043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:36:26.733432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850887+0000","last_change":"2024-09-12T03:03:04.629721+0000","last_active":"2024-09-12T03:04:42.850887+0000","last_peered":"2024-09-12T03:04:42.850887+0000","last_clean":"2024-09-12T03:04:42.850887+0000","last_became_active":"2024-09-12T03:03:04.629362+0000","last_became_peered":"2024-09-12T03:03:04.629362+0000","last_unstale":"2024-09-12T03:04:42.850887+0000","last_undegraded":"2024-09-12T03:04:42.850887+0000","last_fullsized":"2024-09-12T03:04:42.850887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:06:09.902812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459216+0000","last_change":"2024-09-12T03:03:17.521048+0000","last_active":"2024-09-12T03:04:42.459216+0000","last_peered":"2024-09-12T03:04:42.459216+0000","last_clean":"2024-09-12T03:04:42.459216+0000","last_became_active":"2024-09-12T03:03:17.520640+0000","last_became_peered":"2024-09-12T03:03:17.520640+0000","last_unstale":"2024-09-12T03:04:42.459216+0000","last_undegraded":"2024-09-12T03:04:42.459216+0000","last_fullsized":"2024-09-12T03:04:42.459216+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:06.243255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459661+0000","last_change":"2024-09-12T03:03:07.445497+0000","last_active":"2024-09-12T03:04:42.459661+0000","last_peered":"2024-09-12T03:04:42.459661+0000","last_clean":"2024-09-12T03:04:42.459661+0000","last_became_active":"2024-09-12T03:03:07.442245+0000","last_became_peered":"2024-09-12T03:03:07.442245+0000","last_unstale":"2024-09-12T03:04:42.459661+0000","last_undegraded":"2024-09-12T03:04:42.459661+0000","last_fullsized":"2024-09-12T03:04:42.459661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:13.147572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461081+0000","last_change":"2024-09-12T03:03:04.424438+0000","last_active":"2024-09-12T03:04:42.461081+0000","last_peered":"2024-09-12T03:04:42.461081+0000","last_clean":"2024-09-12T03:04:42.461081+0000","last_became_active":"2024-09-12T03:03:04.424078+0000","last_became_peered":"2024-09-12T03:03:04.424078+0000","last_unstale":"2024-09-12T03:04:42.461081+0000","last_undegraded":"2024-09-12T03:04:42.461081+0000","last_fullsized":"2024-09-12T03:04:42.461081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:12:33.090082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457838+0000","last_change":"2024-09-12T03:03:17.532767+0000","last_active":"2024-09-12T03:04:42.457838+0000","last_peered":"2024-09-12T03:04:42.457838+0000","last_clean":"2024-09-12T03:04:42.457838+0000","last_became_active":"2024-09-12T03:03:17.532408+0000","last_became_peered":"2024-09-12T03:03:17.532408+0000","last_unstale":"2024-09-12T03:04:42.457838+0000","last_undegraded":"2024-09-12T03:04:42.457838+0000","last_fullsized":"2024-09-12T03:04:42.457838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:05:29.294605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884833+0000","last_change":"2024-09-12T03:03:07.448931+0000","last_active":"2024-09-12T03:04:42.884833+0000","last_peered":"2024-09-12T03:04:42.884833+0000","last_clean":"2024-09-12T03:04:42.884833+0000","last_became_active":"2024-09-12T03:03:07.447204+0000","last_became_peered":"2024-09-12T03:03:07.447204+0000","last_unstale":"2024-09-12T03:04:42.884833+0000","last_undegraded":"2024-09-12T03:04:42.884833+0000","last_fullsized":"2024-09-12T03:04:42.884833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:04:10.491591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462929+0000","last_change":"2024-09-12T03:03:04.425805+0000","last_active":"2024-09-12T03:04:42.462929+0000","last_peered":"2024-09-12T03:04:42.462929+0000","last_clean":"2024-09-12T03:04:42.462929+0000","last_became_active":"2024-09-12T03:03:04.425469+0000","last_became_peered":"2024-09-12T03:03:04.425469+0000","last_unstale":"2024-09-12T03:04:42.462929+0000","last_undegraded":"2024-09-12T03:04:42.462929+0000","last_fullsized":"2024-09-12T03:04:42.462929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:27.976934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463678+0000","last_change":"2024-09-12T03:03:17.525537+0000","last_active":"2024-09-12T03:04:42.463678+0000","last_peered":"2024-09-12T03:04:42.463678+0000","last_clean":"2024-09-12T03:04:42.463678+0000","last_became_active":"2024-09-12T03:03:17.525030+0000","last_became_peered":"2024-09-12T03:03:17.525030+0000","last_unstale":"2024-09-12T03:04:42.463678+0000","last_undegraded":"2024-09-12T03:04:42.463678+0000","last_fullsized":"2024-09-12T03:04:42.463678+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:38:53.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542918+0000","last_change":"2024-09-12T03:03:07.443712+0000","last_active":"2024-09-12T03:04:42.542918+0000","last_peered":"2024-09-12T03:04:42.542918+0000","last_clean":"2024-09-12T03:04:42.542918+0000","last_became_active":"2024-09-12T03:03:07.443374+0000","last_became_peered":"2024-09-12T03:03:07.443374+0000","last_unstale":"2024-09-12T03:04:42.542918+0000","last_undegraded":"2024-09-12T03:04:42.542918+0000","last_fullsized":"2024-09-12T03:04:42.542918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:04:09.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.461477+0000","last_change":"2024-09-12T03:03:04.427683+0000","last_active":"2024-09-12T03:04:42.461477+0000","last_peered":"2024-09-12T03:04:42.461477+0000","last_clean":"2024-09-12T03:04:42.461477+0000","last_became_active":"2024-09-12T03:03:04.427276+0000","last_became_peered":"2024-09-12T03:03:04.427276+0000","last_unstale":"2024-09-12T03:04:42.461477+0000","last_undegraded":"2024-09-12T03:04:42.461477+0000","last_fullsized":"2024-09-12T03:04:42.461477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:31:44.540369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462141+0000","last_change":"2024-09-12T03:03:41.878110+0000","last_active":"2024-09-12T03:04:42.462141+0000","last_peered":"2024-09-12T03:04:42.462141+0000","last_clean":"2024-09-12T03:04:42.462141+0000","last_became_active":"2024-09-12T03:03:41.877719+0000","last_became_peered":"2024-09-12T03:03:41.877719+0000","last_unstale":"2024-09-12T03:04:42.462141+0000","last_undegraded":"2024-09-12T03:04:42.462141+0000","last_fullsized":"2024-09-12T03:04:42.462141+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:08.312660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542816+0000","last_change":"2024-09-12T03:03:07.454093+0000","last_active":"2024-09-12T03:04:42.542816+0000","last_peered":"2024-09-12T03:04:42.542816+0000","last_clean":"2024-09-12T03:04:42.542816+0000","last_became_active":"2024-09-12T03:03:07.453676+0000","last_became_peered":"2024-09-12T03:03:07.453676+0000","last_unstale":"2024-09-12T03:04:42.542816+0000","last_undegraded":"2024-09-12T03:04:42.542816+0000","last_fullsized":"2024-09-12T03:04:42.542816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:18.045572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458590+0000","last_change":"2024-09-12T03:03:04.425336+0000","last_active":"2024-09-12T03:04:42.458590+0000","last_peered":"2024-09-12T03:04:42.458590+0000","last_clean":"2024-09-12T03:04:42.458590+0000","last_became_active":"2024-09-12T03:03:04.424890+0000","last_became_peered":"2024-09-12T03:03:04.424890+0000","last_unstale":"2024-09-12T03:04:42.458590+0000","last_undegraded":"2024-09-12T03:04:42.458590+0000","last_fullsized":"2024-09-12T03:04:42.458590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:21.443627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884698+0000","last_change":"2024-09-12T03:03:17.535300+0000","last_active":"2024-09-12T03:04:42.884698+0000","last_peered":"2024-09-12T03:04:42.884698+0000","last_clean":"2024-09-12T03:04:42.884698+0000","last_became_active":"2024-09-12T03:03:17.534937+0000","last_became_peered":"2024-09-12T03:03:17.534937+0000","last_unstale":"2024-09-12T03:04:42.884698+0000","last_undegraded":"2024-09-12T03:04:42.884698+0000","last_fullsized":"2024-09-12T03:04:42.884698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:54.874973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718723+0000","last_change":"2024-09-12T03:03:07.453964+0000","last_active":"2024-09-12T03:04:42.718723+0000","last_peered":"2024-09-12T03:04:42.718723+0000","last_clean":"2024-09-12T03:04:42.718723+0000","last_became_active":"2024-09-12T03:03:07.453622+0000","last_became_peered":"2024-09-12T03:03:07.453622+0000","last_unstale":"2024-09-12T03:04:42.718723+0000","last_undegraded":"2024-09-12T03:04:42.718723+0000","last_fullsized":"2024-09-12T03:04:42.718723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:17.452519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'4","reported_seq":112,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.914281+0000","last_change":"2024-09-12T03:03:04.428526+0000","last_active":"2024-09-12T03:04:59.914281+0000","last_peered":"2024-09-12T03:04:59.914281+0000","last_clean":"2024-09-12T03:04:59.914281+0000","last_became_active":"2024-09-12T03:03:04.428221+0000","last_became_peered":"2024-09-12T03:03:04.428221+0000","last_unstale":"2024-09-12T03:04:59.914281+0000","last_undegraded":"2024-09-12T03:04:59.914281+0000","last_fullsized":"2024-09-12T03:04:59.914281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:50.999346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884444+0000","last_change":"2024-09-12T03:03:17.535146+0000","last_active":"2024-09-12T03:04:42.884444+0000","last_peered":"2024-09-12T03:04:42.884444+0000","last_clean":"2024-09-12T03:04:42.884444+0000","last_became_active":"2024-09-12T03:03:17.534677+0000","last_became_peered":"2024-09-12T03:03:17.534677+0000","last_unstale":"2024-09-12T03:04:42.884444+0000","last_undegraded":"2024-09-12T03:04:42.884444+0000","last_fullsized":"2024-09-12T03:04:42.884444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:45:09.874993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884420+0000","last_change":"2024-09-12T03:03:07.431348+0000","last_active":"2024-09-12T03:04:42.884420+0000","last_peered":"2024-09-12T03:04:42.884420+0000","last_clean":"2024-09-12T03:04:42.884420+0000","last_became_active":"2024-09-12T03:03:07.430912+0000","last_became_peered":"2024-09-12T03:03:07.430912+0000","last_unstale":"2024-09-12T03:04:42.884420+0000","last_undegraded":"2024-09-12T03:04:42.884420+0000","last_fullsized":"2024-09-12T03:04:42.884420+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:27.644324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'3","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458311+0000","last_change":"2024-09-12T03:03:42.456245+0000","last_active":"2024-09-12T03:04:42.458311+0000","last_peered":"2024-09-12T03:04:42.458311+0000","last_clean":"2024-09-12T03:04:42.458311+0000","last_became_active":"2024-09-12T03:03:42.455812+0000","last_became_peered":"2024-09-12T03:03:42.455812+0000","last_unstale":"2024-09-12T03:04:42.458311+0000","last_undegraded":"2024-09-12T03:04:42.458311+0000","last_fullsized":"2024-09-12T03:04:42.458311+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:34.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719073+0000","last_change":"2024-09-12T03:03:17.527371+0000","last_active":"2024-09-12T03:04:42.719073+0000","last_peered":"2024-09-12T03:04:42.719073+0000","last_clean":"2024-09-12T03:04:42.719073+0000","last_became_active":"2024-09-12T03:03:17.526818+0000","last_became_peered":"2024-09-12T03:03:17.526818+0000","last_unstale":"2024-09-12T03:04:42.719073+0000","last_undegraded":"2024-09-12T03:04:42.719073+0000","last_fullsized":"2024-09-12T03:04:42.719073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:49.519942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458178+0000","last_change":"2024-09-12T03:03:07.451364+0000","last_active":"2024-09-12T03:04:42.458178+0000","last_peered":"2024-09-12T03:04:42.458178+0000","last_clean":"2024-09-12T03:04:42.458178+0000","last_became_active":"2024-09-12T03:03:07.450964+0000","last_became_peered":"2024-09-12T03:03:07.450964+0000","last_unstale":"2024-09-12T03:04:42.458178+0000","last_undegraded":"2024-09-12T03:04:42.458178+0000","last_fullsized":"2024-09-12T03:04:42.458178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:15.208308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'3","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850880+0000","last_change":"2024-09-12T03:03:04.430485+0000","last_active":"2024-09-12T03:04:42.850880+0000","last_peered":"2024-09-12T03:04:42.850880+0000","last_clean":"2024-09-12T03:04:42.850880+0000","last_became_active":"2024-09-12T03:03:04.430096+0000","last_became_peered":"2024-09-12T03:03:04.430096+0000","last_unstale":"2024-09-12T03:04:42.850880+0000","last_undegraded":"2024-09-12T03:04:42.850880+0000","last_fullsized":"2024-09-12T03:04:42.850880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:26.486223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458445+0000","last_change":"2024-09-12T03:01:48.369350+0000","last_active":"2024-09-12T03:04:42.458445+0000","last_peered":"2024-09-12T03:04:42.458445+0000","last_clean":"2024-09-12T03:04:42.458445+0000","last_became_active":"2024-09-12T03:01:48.369011+0000","last_became_peered":"2024-09-12T03:01:48.369011+0000","last_unstale":"2024-09-12T03:04:42.458445+0000","last_undegraded":"2024-09-12T03:04:42.458445+0000","last_fullsized":"2024-09-12T03:04:42.458445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:58:06.597228+0000","last_clean_scrub_stamp":"2024-09-12T02:58:06.597228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:47:57.348343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458778+0000","last_change":"2024-09-12T03:03:17.541157+0000","last_active":"2024-09-12T03:04:42.458778+0000","last_peered":"2024-09-12T03:04:42.458778+0000","last_clean":"2024-09-12T03:04:42.458778+0000","last_became_active":"2024-09-12T03:03:17.540687+0000","last_became_peered":"2024-09-12T03:03:17.540687+0000","last_unstale":"2024-09-12T03:04:42.458778+0000","last_undegraded":"2024-09-12T03:04:42.458778+0000","last_fullsized":"2024-09-12T03:04:42.458778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:35.072323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458920+0000","last_change":"2024-09-12T03:03:07.445674+0000","last_active":"2024-09-12T03:04:42.458920+0000","last_peered":"2024-09-12T03:04:42.458920+0000","last_clean":"2024-09-12T03:04:42.458920+0000","last_became_active":"2024-09-12T03:03:07.445237+0000","last_became_peered":"2024-09-12T03:03:07.445237+0000","last_unstale":"2024-09-12T03:04:42.458920+0000","last_undegraded":"2024-09-12T03:04:42.458920+0000","last_fullsized":"2024-09-12T03:04:42.458920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:56.876238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459575+0000","last_change":"2024-09-12T03:03:04.629794+0000","last_active":"2024-09-12T03:04:42.459575+0000","last_peered":"2024-09-12T03:04:42.459575+0000","last_clean":"2024-09-12T03:04:42.459575+0000","last_became_active":"2024-09-12T03:03:04.629352+0000","last_became_peered":"2024-09-12T03:03:04.629352+0000","last_unstale":"2024-09-12T03:04:42.459575+0000","last_undegraded":"2024-09-12T03:04:42.459575+0000","last_fullsized":"2024-09-12T03:04:42.459575+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:22:32.168567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064374+0000","last_change":"2024-09-12T03:03:17.525419+0000","last_active":"2024-09-12T03:04:43.064374+0000","last_peered":"2024-09-12T03:04:43.064374+0000","last_clean":"2024-09-12T03:04:43.064374+0000","last_became_active":"2024-09-12T03:03:17.524894+0000","last_became_peered":"2024-09-12T03:03:17.524894+0000","last_unstale":"2024-09-12T03:04:43.064374+0000","last_undegraded":"2024-09-12T03:04:43.064374+0000","last_fullsized":"2024-09-12T03:04:43.064374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:57.441301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458742+0000","last_change":"2024-09-12T03:03:07.447990+0000","last_active":"2024-09-12T03:04:42.458742+0000","last_peered":"2024-09-12T03:04:42.458742+0000","last_clean":"2024-09-12T03:04:42.458742+0000","last_became_active":"2024-09-12T03:03:07.447648+0000","last_became_peered":"2024-09-12T03:03:07.447648+0000","last_unstale":"2024-09-12T03:04:42.458742+0000","last_undegraded":"2024-09-12T03:04:42.458742+0000","last_fullsized":"2024-09-12T03:04:42.458742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:48.639273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719564+0000","last_change":"2024-09-12T03:03:04.421306+0000","last_active":"2024-09-12T03:04:42.719564+0000","last_peered":"2024-09-12T03:04:42.719564+0000","last_clean":"2024-09-12T03:04:42.719564+0000","last_became_active":"2024-09-12T03:03:04.420688+0000","last_became_peered":"2024-09-12T03:03:04.420688+0000","last_unstale":"2024-09-12T03:04:42.719564+0000","last_undegraded":"2024-09-12T03:04:42.719564+0000","last_fullsized":"2024-09-12T03:04:42.719564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:15.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016756+0000","last_change":"2024-09-12T03:03:17.530457+0000","last_active":"2024-09-12T03:04:43.016756+0000","last_peered":"2024-09-12T03:04:43.016756+0000","last_clean":"2024-09-12T03:04:43.016756+0000","last_became_active":"2024-09-12T03:03:17.530044+0000","last_became_peered":"2024-09-12T03:03:17.530044+0000","last_unstale":"2024-09-12T03:04:43.016756+0000","last_undegraded":"2024-09-12T03:04:43.016756+0000","last_fullsized":"2024-09-12T03:04:43.016756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:32:01.304089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459018+0000","last_change":"2024-09-12T03:03:07.445935+0000","last_active":"2024-09-12T03:04:42.459018+0000","last_peered":"2024-09-12T03:04:42.459018+0000","last_clean":"2024-09-12T03:04:42.459018+0000","last_became_active":"2024-09-12T03:03:07.444960+0000","last_became_peered":"2024-09-12T03:03:07.444960+0000","last_unstale":"2024-09-12T03:04:42.459018+0000","last_undegraded":"2024-09-12T03:04:42.459018+0000","last_fullsized":"2024-09-12T03:04:42.459018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:53.246923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884822+0000","last_change":"2024-09-12T03:03:04.424147+0000","last_active":"2024-09-12T03:04:42.884822+0000","last_peered":"2024-09-12T03:04:42.884822+0000","last_clean":"2024-09-12T03:04:42.884822+0000","last_became_active":"2024-09-12T03:03:04.423787+0000","last_became_peered":"2024-09-12T03:03:04.423787+0000","last_unstale":"2024-09-12T03:04:42.884822+0000","last_undegraded":"2024-09-12T03:04:42.884822+0000","last_fullsized":"2024-09-12T03:04:42.884822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:45:48.155556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718464+0000","last_change":"2024-09-12T03:03:17.520974+0000","last_active":"2024-09-12T03:04:42.718464+0000","last_peered":"2024-09-12T03:04:42.718464+0000","last_clean":"2024-09-12T03:04:42.718464+0000","last_became_active":"2024-09-12T03:03:17.520332+0000","last_became_peered":"2024-09-12T03:03:17.520332+0000","last_unstale":"2024-09-12T03:04:42.718464+0000","last_undegraded":"2024-09-12T03:04:42.718464+0000","last_fullsized":"2024-09-12T03:04:42.718464+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:08:45.061197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464537+0000","last_change":"2024-09-12T03:03:42.456589+0000","last_active":"2024-09-12T03:04:42.464537+0000","last_peered":"2024-09-12T03:04:42.464537+0000","last_clean":"2024-09-12T03:04:42.464537+0000","last_became_active":"2024-09-12T03:03:42.456147+0000","last_became_peered":"2024-09-12T03:03:42.456147+0000","last_unstale":"2024-09-12T03:04:42.464537+0000","last_undegraded":"2024-09-12T03:04:42.464537+0000","last_fullsized":"2024-09-12T03:04:42.464537+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:58.009928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463226+0000","last_change":"2024-09-12T03:03:04.420471+0000","last_active":"2024-09-12T03:04:42.463226+0000","last_peered":"2024-09-12T03:04:42.463226+0000","last_clean":"2024-09-12T03:04:42.463226+0000","last_became_active":"2024-09-12T03:03:04.420093+0000","last_became_peered":"2024-09-12T03:03:04.420093+0000","last_unstale":"2024-09-12T03:04:42.463226+0000","last_undegraded":"2024-09-12T03:04:42.463226+0000","last_fullsized":"2024-09-12T03:04:42.463226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:23.772561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718669+0000","last_change":"2024-09-12T03:03:17.528073+0000","last_active":"2024-09-12T03:04:42.718669+0000","last_peered":"2024-09-12T03:04:42.718669+0000","last_clean":"2024-09-12T03:04:42.718669+0000","last_became_active":"2024-09-12T03:03:17.527654+0000","last_became_peered":"2024-09-12T03:03:17.527654+0000","last_unstale":"2024-09-12T03:04:42.718669+0000","last_undegraded":"2024-09-12T03:04:42.718669+0000","last_fullsized":"2024-09-12T03:04:42.718669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:45.806736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718749+0000","last_change":"2024-09-12T03:03:07.455305+0000","last_active":"2024-09-12T03:04:42.718749+0000","last_peered":"2024-09-12T03:04:42.718749+0000","last_clean":"2024-09-12T03:04:42.718749+0000","last_became_active":"2024-09-12T03:03:07.454957+0000","last_became_peered":"2024-09-12T03:03:07.454957+0000","last_unstale":"2024-09-12T03:04:42.718749+0000","last_undegraded":"2024-09-12T03:04:42.718749+0000","last_fullsized":"2024-09-12T03:04:42.718749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:24:07.152100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'15","reported_seq":59,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462091+0000","last_change":"2024-09-12T03:03:04.424531+0000","last_active":"2024-09-12T03:04:42.462091+0000","last_peered":"2024-09-12T03:04:42.462091+0000","last_clean":"2024-09-12T03:04:42.462091+0000","last_became_active":"2024-09-12T03:03:04.424230+0000","last_became_peered":"2024-09-12T03:03:04.424230+0000","last_unstale":"2024-09-12T03:04:42.462091+0000","last_undegraded":"2024-09-12T03:04:42.462091+0000","last_fullsized":"2024-09-12T03:04:42.462091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:01.392262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884779+0000","last_change":"2024-09-12T03:03:17.532131+0000","last_active":"2024-09-12T03:04:42.884779+0000","last_peered":"2024-09-12T03:04:42.884779+0000","last_clean":"2024-09-12T03:04:42.884779+0000","last_became_active":"2024-09-12T03:03:17.531711+0000","last_became_peered":"2024-09-12T03:03:17.531711+0000","last_unstale":"2024-09-12T03:04:42.884779+0000","last_undegraded":"2024-09-12T03:04:42.884779+0000","last_fullsized":"2024-09-12T03:04:42.884779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:15.880954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458191+0000","last_change":"2024-09-12T03:03:07.449101+0000","last_active":"2024-09-12T03:04:42.458191+0000","last_peered":"2024-09-12T03:04:42.458191+0000","last_clean":"2024-09-12T03:04:42.458191+0000","last_became_active":"2024-09-12T03:03:07.448719+0000","last_became_peered":"2024-09-12T03:03:07.448719+0000","last_unstale":"2024-09-12T03:04:42.458191+0000","last_undegraded":"2024-09-12T03:04:42.458191+0000","last_fullsized":"2024-09-12T03:04:42.458191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:29:45.400939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'3","reported_seq":114,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.915279+0000","last_change":"2024-09-12T03:03:04.422174+0000","last_active":"2024-09-12T03:04:59.915279+0000","last_peered":"2024-09-12T03:04:59.915279+0000","last_clean":"2024-09-12T03:04:59.915279+0000","last_became_active":"2024-09-12T03:03:04.421771+0000","last_became_peered":"2024-09-12T03:03:04.421771+0000","last_unstale":"2024-09-12T03:04:59.915279+0000","last_undegraded":"2024-09-12T03:04:59.915279+0000","last_fullsized":"2024-09-12T03:04:59.915279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:38.149590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460718+0000","last_change":"2024-09-12T03:03:17.535123+0000","last_active":"2024-09-12T03:04:42.460718+0000","last_peered":"2024-09-12T03:04:42.460718+0000","last_clean":"2024-09-12T03:04:42.460718+0000","last_became_active":"2024-09-12T03:03:17.534218+0000","last_became_peered":"2024-09-12T03:03:17.534218+0000","last_unstale":"2024-09-12T03:04:42.460718+0000","last_undegraded":"2024-09-12T03:04:42.460718+0000","last_fullsized":"2024-09-12T03:04:42.460718+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:55.870690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458503+0000","last_change":"2024-09-12T03:03:07.453736+0000","last_active":"2024-09-12T03:04:42.458503+0000","last_peered":"2024-09-12T03:04:42.458503+0000","last_clean":"2024-09-12T03:04:42.458503+0000","last_became_active":"2024-09-12T03:03:07.453404+0000","last_became_peered":"2024-09-12T03:03:07.453404+0000","last_unstale":"2024-09-12T03:04:42.458503+0000","last_undegraded":"2024-09-12T03:04:42.458503+0000","last_fullsized":"2024-09-12T03:04:42.458503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:57.220001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719393+0000","last_change":"2024-09-12T03:03:04.419663+0000","last_active":"2024-09-12T03:04:42.719393+0000","last_peered":"2024-09-12T03:04:42.719393+0000","last_clean":"2024-09-12T03:04:42.719393+0000","last_became_active":"2024-09-12T03:03:04.419119+0000","last_became_peered":"2024-09-12T03:03:04.419119+0000","last_unstale":"2024-09-12T03:04:42.719393+0000","last_undegraded":"2024-09-12T03:04:42.719393+0000","last_fullsized":"2024-09-12T03:04:42.719393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:44:11.809683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463824+0000","last_change":"2024-09-12T03:03:17.532686+0000","last_active":"2024-09-12T03:04:42.463824+0000","last_peered":"2024-09-12T03:04:42.463824+0000","last_clean":"2024-09-12T03:04:42.463824+0000","last_became_active":"2024-09-12T03:03:17.532234+0000","last_became_peered":"2024-09-12T03:03:17.532234+0000","last_unstale":"2024-09-12T03:04:42.463824+0000","last_undegraded":"2024-09-12T03:04:42.463824+0000","last_fullsized":"2024-09-12T03:04:42.463824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:54:49.304077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884316+0000","last_change":"2024-09-12T03:03:07.441911+0000","last_active":"2024-09-12T03:04:42.884316+0000","last_peered":"2024-09-12T03:04:42.884316+0000","last_clean":"2024-09-12T03:04:42.884316+0000","last_became_active":"2024-09-12T03:03:07.441323+0000","last_became_peered":"2024-09-12T03:03:07.441323+0000","last_unstale":"2024-09-12T03:04:42.884316+0000","last_undegraded":"2024-09-12T03:04:42.884316+0000","last_fullsized":"2024-09-12T03:04:42.884316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:55:03.830843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'6","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459398+0000","last_change":"2024-09-12T03:03:04.423648+0000","last_active":"2024-09-12T03:04:42.459398+0000","last_peered":"2024-09-12T03:04:42.459398+0000","last_clean":"2024-09-12T03:04:42.459398+0000","last_became_active":"2024-09-12T03:03:04.423274+0000","last_became_peered":"2024-09-12T03:03:04.423274+0000","last_unstale":"2024-09-12T03:04:42.459398+0000","last_undegraded":"2024-09-12T03:04:42.459398+0000","last_fullsized":"2024-09-12T03:04:42.459398+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:25:50.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719426+0000","last_change":"2024-09-12T03:03:17.522286+0000","last_active":"2024-09-12T03:04:42.719426+0000","last_peered":"2024-09-12T03:04:42.719426+0000","last_clean":"2024-09-12T03:04:42.719426+0000","last_became_active":"2024-09-12T03:03:17.521909+0000","last_became_peered":"2024-09-12T03:03:17.521909+0000","last_unstale":"2024-09-12T03:04:42.719426+0000","last_undegraded":"2024-09-12T03:04:42.719426+0000","last_fullsized":"2024-09-12T03:04:42.719426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:13.093884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462495+0000","last_change":"2024-09-12T03:03:07.445445+0000","last_active":"2024-09-12T03:04:42.462495+0000","last_peered":"2024-09-12T03:04:42.462495+0000","last_clean":"2024-09-12T03:04:42.462495+0000","last_became_active":"2024-09-12T03:03:07.445086+0000","last_became_peered":"2024-09-12T03:03:07.445086+0000","last_unstale":"2024-09-12T03:04:42.462495+0000","last_undegraded":"2024-09-12T03:04:42.462495+0000","last_fullsized":"2024-09-12T03:04:42.462495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:10:01.046168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'32","reported_seq":212,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.902976+0000","last_change":"2024-09-12T03:03:04.425545+0000","last_active":"2024-09-12T03:04:59.902976+0000","last_peered":"2024-09-12T03:04:59.902976+0000","last_clean":"2024-09-12T03:04:59.902976+0000","last_became_active":"2024-09-12T03:03:04.425141+0000","last_became_peered":"2024-09-12T03:03:04.425141+0000","last_unstale":"2024-09-12T03:04:59.902976+0000","last_undegraded":"2024-09-12T03:04:59.902976+0000","last_fullsized":"2024-09-12T03:04:59.902976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:41:54.548787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62661,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":136,"num_read_kb":129,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460282+0000","last_change":"2024-09-12T03:03:17.538157+0000","last_active":"2024-09-12T03:04:42.460282+0000","last_peered":"2024-09-12T03:04:42.460282+0000","last_clean":"2024-09-12T03:04:42.460282+0000","last_became_active":"2024-09-12T03:03:17.537194+0000","last_became_peered":"2024-09-12T03:03:17.537194+0000","last_unstale":"2024-09-12T03:04:42.460282+0000","last_undegraded":"2024-09-12T03:04:42.460282+0000","last_fullsized":"2024-09-12T03:04:42.460282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:32:00.343037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064669+0000","last_change":"2024-09-12T03:03:07.446470+0000","last_active":"2024-09-12T03:04:43.064669+0000","last_peered":"2024-09-12T03:04:43.064669+0000","last_clean":"2024-09-12T03:04:43.064669+0000","last_became_active":"2024-09-12T03:03:07.445975+0000","last_became_peered":"2024-09-12T03:03:07.445975+0000","last_unstale":"2024-09-12T03:04:43.064669+0000","last_undegraded":"2024-09-12T03:04:43.064669+0000","last_fullsized":"2024-09-12T03:04:43.064669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:33:47.795117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719517+0000","last_change":"2024-09-12T03:03:04.421833+0000","last_active":"2024-09-12T03:04:42.719517+0000","last_peered":"2024-09-12T03:04:42.719517+0000","last_clean":"2024-09-12T03:04:42.719517+0000","last_became_active":"2024-09-12T03:03:04.421439+0000","last_became_peered":"2024-09-12T03:03:04.421439+0000","last_unstale":"2024-09-12T03:04:42.719517+0000","last_undegraded":"2024-09-12T03:04:42.719517+0000","last_fullsized":"2024-09-12T03:04:42.719517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.850564+0000","last_change":"2024-09-12T03:03:17.541327+0000","last_active":"2024-09-12T03:04:42.850564+0000","last_peered":"2024-09-12T03:04:42.850564+0000","last_clean":"2024-09-12T03:04:42.850564+0000","last_became_active":"2024-09-12T03:03:17.540959+0000","last_became_peered":"2024-09-12T03:03:17.540959+0000","last_unstale":"2024-09-12T03:04:42.850564+0000","last_undegraded":"2024-09-12T03:04:42.850564+0000","last_fullsized":"2024-09-12T03:04:42.850564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:06:16.833511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.017157+0000","last_change":"2024-09-12T03:03:07.458806+0000","last_active":"2024-09-12T03:04:43.017157+0000","last_peered":"2024-09-12T03:04:43.017157+0000","last_clean":"2024-09-12T03:04:43.017157+0000","last_became_active":"2024-09-12T03:03:07.458501+0000","last_became_peered":"2024-09-12T03:03:07.458501+0000","last_unstale":"2024-09-12T03:04:43.017157+0000","last_undegraded":"2024-09-12T03:04:43.017157+0000","last_fullsized":"2024-09-12T03:04:43.017157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:54.608552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'2","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458906+0000","last_change":"2024-09-12T03:03:41.676350+0000","last_active":"2024-09-12T03:04:42.458906+0000","last_peered":"2024-09-12T03:04:42.458906+0000","last_clean":"2024-09-12T03:04:42.458906+0000","last_became_active":"2024-09-12T03:03:41.675935+0000","last_became_peered":"2024-09-12T03:03:41.675935+0000","last_unstale":"2024-09-12T03:04:42.458906+0000","last_undegraded":"2024-09-12T03:04:42.458906+0000","last_fullsized":"2024-09-12T03:04:42.458906+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:11.305186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460124+0000","last_change":"2024-09-12T03:03:17.533670+0000","last_active":"2024-09-12T03:04:42.460124+0000","last_peered":"2024-09-12T03:04:42.460124+0000","last_clean":"2024-09-12T03:04:42.460124+0000","last_became_active":"2024-09-12T03:03:17.530516+0000","last_became_peered":"2024-09-12T03:03:17.530516+0000","last_unstale":"2024-09-12T03:04:42.460124+0000","last_undegraded":"2024-09-12T03:04:42.460124+0000","last_fullsized":"2024-09-12T03:04:42.460124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:02.228503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064753+0000","last_change":"2024-09-12T03:03:07.448760+0000","last_active":"2024-09-12T03:04:43.064753+0000","last_peered":"2024-09-12T03:04:43.064753+0000","last_clean":"2024-09-12T03:04:43.064753+0000","last_became_active":"2024-09-12T03:03:07.447978+0000","last_became_peered":"2024-09-12T03:03:07.447978+0000","last_unstale":"2024-09-12T03:04:43.064753+0000","last_undegraded":"2024-09-12T03:04:43.064753+0000","last_fullsized":"2024-09-12T03:04:43.064753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:51.319881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'3","reported_seq":111,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:58.901725+0000","last_change":"2024-09-12T03:03:04.427752+0000","last_active":"2024-09-12T03:04:58.901725+0000","last_peered":"2024-09-12T03:04:58.901725+0000","last_clean":"2024-09-12T03:04:58.901725+0000","last_became_active":"2024-09-12T03:03:04.427327+0000","last_became_peered":"2024-09-12T03:03:04.427327+0000","last_unstale":"2024-09-12T03:04:58.901725+0000","last_undegraded":"2024-09-12T03:04:58.901725+0000","last_fullsized":"2024-09-12T03:04:58.901725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:51.271870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457963+0000","last_change":"2024-09-12T03:03:17.520463+0000","last_active":"2024-09-12T03:04:42.457963+0000","last_peered":"2024-09-12T03:04:42.457963+0000","last_clean":"2024-09-12T03:04:42.457963+0000","last_became_active":"2024-09-12T03:03:17.520081+0000","last_became_peered":"2024-09-12T03:03:17.520081+0000","last_unstale":"2024-09-12T03:04:42.457963+0000","last_undegraded":"2024-09-12T03:04:42.457963+0000","last_fullsized":"2024-09-12T03:04:42.457963+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:56.707115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016916+0000","last_change":"2024-09-12T03:03:17.532373+0000","last_active":"2024-09-12T03:04:43.016916+0000","last_peered":"2024-09-12T03:04:43.016916+0000","last_clean":"2024-09-12T03:04:43.016916+0000","last_became_active":"2024-09-12T03:03:17.525425+0000","last_became_peered":"2024-09-12T03:03:17.525425+0000","last_unstale":"2024-09-12T03:04:43.016916+0000","last_undegraded":"2024-09-12T03:04:43.016916+0000","last_fullsized":"2024-09-12T03:04:43.016916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:12:29.811678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460734+0000","last_change":"2024-09-12T03:03:07.445936+0000","last_active":"2024-09-12T03:04:42.460734+0000","last_peered":"2024-09-12T03:04:42.460734+0000","last_clean":"2024-09-12T03:04:42.460734+0000","last_became_active":"2024-09-12T03:03:07.445643+0000","last_became_peered":"2024-09-12T03:03:07.445643+0000","last_unstale":"2024-09-12T03:04:42.460734+0000","last_undegraded":"2024-09-12T03:04:42.460734+0000","last_fullsized":"2024-09-12T03:04:42.460734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:55:07.477841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718841+0000","last_change":"2024-09-12T03:03:04.424947+0000","last_active":"2024-09-12T03:04:42.718841+0000","last_peered":"2024-09-12T03:04:42.718841+0000","last_clean":"2024-09-12T03:04:42.718841+0000","last_became_active":"2024-09-12T03:03:04.424297+0000","last_became_peered":"2024-09-12T03:03:04.424297+0000","last_unstale":"2024-09-12T03:04:42.718841+0000","last_undegraded":"2024-09-12T03:04:42.718841+0000","last_fullsized":"2024-09-12T03:04:42.718841+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:11.187945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464303+0000","last_change":"2024-09-12T03:03:17.530616+0000","last_active":"2024-09-12T03:04:42.464303+0000","last_peered":"2024-09-12T03:04:42.464303+0000","last_clean":"2024-09-12T03:04:42.464303+0000","last_became_active":"2024-09-12T03:03:17.530146+0000","last_became_peered":"2024-09-12T03:03:17.530146+0000","last_unstale":"2024-09-12T03:04:42.464303+0000","last_undegraded":"2024-09-12T03:04:42.464303+0000","last_fullsized":"2024-09-12T03:04:42.464303+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:26.892311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457982+0000","last_change":"2024-09-12T03:03:07.448084+0000","last_active":"2024-09-12T03:04:42.457982+0000","last_peered":"2024-09-12T03:04:42.457982+0000","last_clean":"2024-09-12T03:04:42.457982+0000","last_became_active":"2024-09-12T03:03:07.447363+0000","last_became_peered":"2024-09-12T03:03:07.447363+0000","last_unstale":"2024-09-12T03:04:42.457982+0000","last_undegraded":"2024-09-12T03:04:42.457982+0000","last_fullsized":"2024-09-12T03:04:42.457982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:07.451892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884163+0000","last_change":"2024-09-12T03:03:04.430371+0000","last_active":"2024-09-12T03:04:42.884163+0000","last_peered":"2024-09-12T03:04:42.884163+0000","last_clean":"2024-09-12T03:04:42.884163+0000","last_became_active":"2024-09-12T03:03:04.429811+0000","last_became_peered":"2024-09-12T03:03:04.429811+0000","last_unstale":"2024-09-12T03:04:42.884163+0000","last_undegraded":"2024-09-12T03:04:42.884163+0000","last_fullsized":"2024-09-12T03:04:42.884163+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:07.443242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542797+0000","last_change":"2024-09-12T03:03:04.422909+0000","last_active":"2024-09-12T03:04:42.542797+0000","last_peered":"2024-09-12T03:04:42.542797+0000","last_clean":"2024-09-12T03:04:42.542797+0000","last_became_active":"2024-09-12T03:03:04.422476+0000","last_became_peered":"2024-09-12T03:03:04.422476+0000","last_unstale":"2024-09-12T03:04:42.542797+0000","last_undegraded":"2024-09-12T03:04:42.542797+0000","last_fullsized":"2024-09-12T03:04:42.542797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:27:46.154499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.719190+0000","last_change":"2024-09-12T03:03:07.454873+0000","last_active":"2024-09-12T03:04:42.719190+0000","last_peered":"2024-09-12T03:04:42.719190+0000","last_clean":"2024-09-12T03:04:42.719190+0000","last_became_active":"2024-09-12T03:03:07.454521+0000","last_became_peered":"2024-09-12T03:03:07.454521+0000","last_unstale":"2024-09-12T03:04:42.719190+0000","last_undegraded":"2024-09-12T03:04:42.719190+0000","last_fullsized":"2024-09-12T03:04:42.719190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:01.710362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459234+0000","last_change":"2024-09-12T03:03:42.456398+0000","last_active":"2024-09-12T03:04:42.459234+0000","last_peered":"2024-09-12T03:04:42.459234+0000","last_clean":"2024-09-12T03:04:42.459234+0000","last_became_active":"2024-09-12T03:03:42.455878+0000","last_became_peered":"2024-09-12T03:03:42.455878+0000","last_unstale":"2024-09-12T03:04:42.459234+0000","last_undegraded":"2024-09-12T03:04:42.459234+0000","last_fullsized":"2024-09-12T03:04:42.459234+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:31.760018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:59.903504+0000","last_change":"2024-09-12T03:03:04.424039+0000","last_active":"2024-09-12T03:04:59.903504+0000","last_peered":"2024-09-12T03:04:59.903504+0000","last_clean":"2024-09-12T03:04:59.903504+0000","last_became_active":"2024-09-12T03:03:04.422184+0000","last_became_peered":"2024-09-12T03:03:04.422184+0000","last_unstale":"2024-09-12T03:04:59.903504+0000","last_undegraded":"2024-09-12T03:04:59.903504+0000","last_fullsized":"2024-09-12T03:04:59.903504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:03.395056+0000","last_clean_scrub_stamp":"2024-09-12T03:03:03.395056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:07.108262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464950+0000","last_change":"2024-09-12T03:04:42.464950+0000","last_active":"2024-09-12T03:04:42.464950+0000","last_peered":"2024-09-12T03:04:42.464950+0000","last_clean":"2024-09-12T03:04:42.464950+0000","last_became_active":"2024-09-12T03:04:42.464561+0000","last_became_peered":"2024-09-12T03:04:42.464561+0000","last_unstale":"2024-09-12T03:04:42.464950+0000","last_undegraded":"2024-09-12T03:04:42.464950+0000","last_fullsized":"2024-09-12T03:04:42.464950+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:23:32.665086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459597+0000","last_change":"2024-09-12T03:03:17.521454+0000","last_active":"2024-09-12T03:04:42.459597+0000","last_peered":"2024-09-12T03:04:42.459597+0000","last_clean":"2024-09-12T03:04:42.459597+0000","last_became_active":"2024-09-12T03:03:17.521053+0000","last_became_peered":"2024-09-12T03:03:17.521053+0000","last_unstale":"2024-09-12T03:04:42.459597+0000","last_undegraded":"2024-09-12T03:04:42.459597+0000","last_fullsized":"2024-09-12T03:04:42.459597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:16.484568+0000","last_clean_scrub_stamp":"2024-09-12T03:03:16.484568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:03:57.088465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.463680+0000","last_change":"2024-09-12T03:03:07.451140+0000","last_active":"2024-09-12T03:04:42.463680+0000","last_peered":"2024-09-12T03:04:42.463680+0000","last_clean":"2024-09-12T03:04:42.463680+0000","last_became_active":"2024-09-12T03:03:07.450775+0000","last_became_peered":"2024-09-12T03:03:07.450775+0000","last_unstale":"2024-09-12T03:04:42.463680+0000","last_undegraded":"2024-09-12T03:04:42.463680+0000","last_fullsized":"2024-09-12T03:04:42.463680+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:28:12.131080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459507+0000","last_change":"2024-09-12T03:03:07.445602+0000","last_active":"2024-09-12T03:04:42.459507+0000","last_peered":"2024-09-12T03:04:42.459507+0000","last_clean":"2024-09-12T03:04:42.459507+0000","last_became_active":"2024-09-12T03:03:07.445210+0000","last_became_peered":"2024-09-12T03:03:07.445210+0000","last_unstale":"2024-09-12T03:04:42.459507+0000","last_undegraded":"2024-09-12T03:04:42.459507+0000","last_fullsized":"2024-09-12T03:04:42.459507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:48.044322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459097+0000","last_change":"2024-09-12T03:03:07.453006+0000","last_active":"2024-09-12T03:04:42.459097+0000","last_peered":"2024-09-12T03:04:42.459097+0000","last_clean":"2024-09-12T03:04:42.459097+0000","last_became_active":"2024-09-12T03:03:07.452608+0000","last_became_peered":"2024-09-12T03:03:07.452608+0000","last_unstale":"2024-09-12T03:04:42.459097+0000","last_undegraded":"2024-09-12T03:04:42.459097+0000","last_fullsized":"2024-09-12T03:04:42.459097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:22.443868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016781+0000","last_change":"2024-09-12T03:03:07.457284+0000","last_active":"2024-09-12T03:04:43.016781+0000","last_peered":"2024-09-12T03:04:43.016781+0000","last_clean":"2024-09-12T03:04:43.016781+0000","last_became_active":"2024-09-12T03:03:07.456964+0000","last_became_peered":"2024-09-12T03:03:07.456964+0000","last_unstale":"2024-09-12T03:04:43.016781+0000","last_undegraded":"2024-09-12T03:04:43.016781+0000","last_fullsized":"2024-09-12T03:04:43.016781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:53.525035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884816+0000","last_change":"2024-09-12T03:03:07.451622+0000","last_active":"2024-09-12T03:04:42.884816+0000","last_peered":"2024-09-12T03:04:42.884816+0000","last_clean":"2024-09-12T03:04:42.884816+0000","last_became_active":"2024-09-12T03:03:07.451240+0000","last_became_peered":"2024-09-12T03:03:07.451240+0000","last_unstale":"2024-09-12T03:04:42.884816+0000","last_undegraded":"2024-09-12T03:04:42.884816+0000","last_fullsized":"2024-09-12T03:04:42.884816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:17.357741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.465641+0000","last_change":"2024-09-12T03:04:42.465641+0000","last_active":"2024-09-12T03:04:42.465641+0000","last_peered":"2024-09-12T03:04:42.465641+0000","last_clean":"2024-09-12T03:04:42.465641+0000","last_became_active":"2024-09-12T03:04:42.465303+0000","last_became_peered":"2024-09-12T03:04:42.465303+0000","last_unstale":"2024-09-12T03:04:42.465641+0000","last_undegraded":"2024-09-12T03:04:42.465641+0000","last_fullsized":"2024-09-12T03:04:42.465641+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:31.674526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459670+0000","last_change":"2024-09-12T03:03:07.444125+0000","last_active":"2024-09-12T03:04:42.459670+0000","last_peered":"2024-09-12T03:04:42.459670+0000","last_clean":"2024-09-12T03:04:42.459670+0000","last_became_active":"2024-09-12T03:03:07.443799+0000","last_became_peered":"2024-09-12T03:03:07.443799+0000","last_unstale":"2024-09-12T03:04:42.459670+0000","last_undegraded":"2024-09-12T03:04:42.459670+0000","last_fullsized":"2024-09-12T03:04:42.459670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:28:49.573684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459566+0000","last_change":"2024-09-12T03:03:07.446562+0000","last_active":"2024-09-12T03:04:42.459566+0000","last_peered":"2024-09-12T03:04:42.459566+0000","last_clean":"2024-09-12T03:04:42.459566+0000","last_became_active":"2024-09-12T03:03:07.445539+0000","last_became_peered":"2024-09-12T03:03:07.445539+0000","last_unstale":"2024-09-12T03:04:42.459566+0000","last_undegraded":"2024-09-12T03:04:42.459566+0000","last_fullsized":"2024-09-12T03:04:42.459566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:22:06.939290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.718642+0000","last_change":"2024-09-12T03:03:07.444899+0000","last_active":"2024-09-12T03:04:42.718642+0000","last_peered":"2024-09-12T03:04:42.718642+0000","last_clean":"2024-09-12T03:04:42.718642+0000","last_became_active":"2024-09-12T03:03:07.443309+0000","last_became_peered":"2024-09-12T03:03:07.443309+0000","last_unstale":"2024-09-12T03:04:42.718642+0000","last_undegraded":"2024-09-12T03:04:42.718642+0000","last_fullsized":"2024-09-12T03:04:42.718642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:06.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.466478+0000","last_change":"2024-09-12T03:04:42.465614+0000","last_active":"2024-09-12T03:04:42.466478+0000","last_peered":"2024-09-12T03:04:42.466478+0000","last_clean":"2024-09-12T03:04:42.466478+0000","last_became_active":"2024-09-12T03:04:42.465250+0000","last_became_peered":"2024-09-12T03:04:42.465250+0000","last_unstale":"2024-09-12T03:04:42.466478+0000","last_undegraded":"2024-09-12T03:04:42.466478+0000","last_fullsized":"2024-09-12T03:04:42.466478+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:29.305821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064550+0000","last_change":"2024-09-12T03:03:07.445924+0000","last_active":"2024-09-12T03:04:43.064550+0000","last_peered":"2024-09-12T03:04:43.064550+0000","last_clean":"2024-09-12T03:04:43.064550+0000","last_became_active":"2024-09-12T03:03:07.445449+0000","last_became_peered":"2024-09-12T03:03:07.445449+0000","last_unstale":"2024-09-12T03:04:43.064550+0000","last_undegraded":"2024-09-12T03:04:43.064550+0000","last_fullsized":"2024-09-12T03:04:43.064550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:42.040044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884569+0000","last_change":"2024-09-12T03:03:07.449653+0000","last_active":"2024-09-12T03:04:42.884569+0000","last_peered":"2024-09-12T03:04:42.884569+0000","last_clean":"2024-09-12T03:04:42.884569+0000","last_became_active":"2024-09-12T03:03:07.445691+0000","last_became_peered":"2024-09-12T03:03:07.445691+0000","last_unstale":"2024-09-12T03:04:42.884569+0000","last_undegraded":"2024-09-12T03:04:42.884569+0000","last_fullsized":"2024-09-12T03:04:42.884569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:53.210238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.465695+0000","last_change":"2024-09-12T03:04:42.465695+0000","last_active":"2024-09-12T03:04:42.465695+0000","last_peered":"2024-09-12T03:04:42.465695+0000","last_clean":"2024-09-12T03:04:42.465695+0000","last_became_active":"2024-09-12T03:04:42.465338+0000","last_became_peered":"2024-09-12T03:04:42.465338+0000","last_unstale":"2024-09-12T03:04:42.465695+0000","last_undegraded":"2024-09-12T03:04:42.465695+0000","last_fullsized":"2024-09-12T03:04:42.465695+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:01:22.516114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.459611+0000","last_change":"2024-09-12T03:03:07.441645+0000","last_active":"2024-09-12T03:04:42.459611+0000","last_peered":"2024-09-12T03:04:42.459611+0000","last_clean":"2024-09-12T03:04:42.459611+0000","last_became_active":"2024-09-12T03:03:07.441191+0000","last_became_peered":"2024-09-12T03:03:07.441191+0000","last_unstale":"2024-09-12T03:04:42.459611+0000","last_undegraded":"2024-09-12T03:04:42.459611+0000","last_fullsized":"2024-09-12T03:04:42.459611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:31:40.873214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.460768+0000","last_change":"2024-09-12T03:03:07.446707+0000","last_active":"2024-09-12T03:04:42.460768+0000","last_peered":"2024-09-12T03:04:42.460768+0000","last_clean":"2024-09-12T03:04:42.460768+0000","last_became_active":"2024-09-12T03:03:07.446301+0000","last_became_peered":"2024-09-12T03:03:07.446301+0000","last_unstale":"2024-09-12T03:04:42.460768+0000","last_undegraded":"2024-09-12T03:04:42.460768+0000","last_fullsized":"2024-09-12T03:04:42.460768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:52:30.610892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464637+0000","last_change":"2024-09-12T03:03:07.449478+0000","last_active":"2024-09-12T03:04:42.464637+0000","last_peered":"2024-09-12T03:04:42.464637+0000","last_clean":"2024-09-12T03:04:42.464637+0000","last_became_active":"2024-09-12T03:03:07.448830+0000","last_became_peered":"2024-09-12T03:03:07.448830+0000","last_unstale":"2024-09-12T03:04:42.464637+0000","last_undegraded":"2024-09-12T03:04:42.464637+0000","last_fullsized":"2024-09-12T03:04:42.464637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:55.411006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064337+0000","last_change":"2024-09-12T03:03:07.449866+0000","last_active":"2024-09-12T03:04:43.064337+0000","last_peered":"2024-09-12T03:04:43.064337+0000","last_clean":"2024-09-12T03:04:43.064337+0000","last_became_active":"2024-09-12T03:03:07.448934+0000","last_became_peered":"2024-09-12T03:03:07.448934+0000","last_unstale":"2024-09-12T03:04:43.064337+0000","last_undegraded":"2024-09-12T03:04:43.064337+0000","last_fullsized":"2024-09-12T03:04:43.064337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:04:24.469912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016515+0000","last_change":"2024-09-12T03:03:41.673404+0000","last_active":"2024-09-12T03:04:43.016515+0000","last_peered":"2024-09-12T03:04:43.016515+0000","last_clean":"2024-09-12T03:04:43.016515+0000","last_became_active":"2024-09-12T03:03:41.673022+0000","last_became_peered":"2024-09-12T03:03:41.673022+0000","last_unstale":"2024-09-12T03:04:43.016515+0000","last_undegraded":"2024-09-12T03:04:43.016515+0000","last_fullsized":"2024-09-12T03:04:43.016515+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:44.083090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016583+0000","last_change":"2024-09-12T03:03:07.455058+0000","last_active":"2024-09-12T03:04:43.016583+0000","last_peered":"2024-09-12T03:04:43.016583+0000","last_clean":"2024-09-12T03:04:43.016583+0000","last_became_active":"2024-09-12T03:03:07.449331+0000","last_became_peered":"2024-09-12T03:03:07.449331+0000","last_unstale":"2024-09-12T03:04:43.016583+0000","last_undegraded":"2024-09-12T03:04:43.016583+0000","last_fullsized":"2024-09-12T03:04:43.016583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:30.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.884540+0000","last_change":"2024-09-12T03:03:07.447193+0000","last_active":"2024-09-12T03:04:42.884540+0000","last_peered":"2024-09-12T03:04:42.884540+0000","last_clean":"2024-09-12T03:04:42.884540+0000","last_became_active":"2024-09-12T03:03:07.445206+0000","last_became_peered":"2024-09-12T03:03:07.445206+0000","last_unstale":"2024-09-12T03:04:42.884540+0000","last_undegraded":"2024-09-12T03:04:42.884540+0000","last_fullsized":"2024-09-12T03:04:42.884540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:36:28.919293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.851095+0000","last_change":"2024-09-12T03:03:07.442621+0000","last_active":"2024-09-12T03:04:42.851095+0000","last_peered":"2024-09-12T03:04:42.851095+0000","last_clean":"2024-09-12T03:04:42.851095+0000","last_became_active":"2024-09-12T03:03:07.441124+0000","last_became_peered":"2024-09-12T03:03:07.441124+0000","last_unstale":"2024-09-12T03:04:42.851095+0000","last_undegraded":"2024-09-12T03:04:42.851095+0000","last_fullsized":"2024-09-12T03:04:42.851095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:08.442627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458916+0000","last_change":"2024-09-12T03:03:07.443619+0000","last_active":"2024-09-12T03:04:42.458916+0000","last_peered":"2024-09-12T03:04:42.458916+0000","last_clean":"2024-09-12T03:04:42.458916+0000","last_became_active":"2024-09-12T03:03:07.443290+0000","last_became_peered":"2024-09-12T03:03:07.443290+0000","last_unstale":"2024-09-12T03:04:42.458916+0000","last_undegraded":"2024-09-12T03:04:42.458916+0000","last_fullsized":"2024-09-12T03:04:42.458916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:43.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.543277+0000","last_change":"2024-09-12T03:03:07.454164+0000","last_active":"2024-09-12T03:04:42.543277+0000","last_peered":"2024-09-12T03:04:42.543277+0000","last_clean":"2024-09-12T03:04:42.543277+0000","last_became_active":"2024-09-12T03:03:07.453835+0000","last_became_peered":"2024-09-12T03:03:07.453835+0000","last_unstale":"2024-09-12T03:04:42.543277+0000","last_undegraded":"2024-09-12T03:04:42.543277+0000","last_fullsized":"2024-09-12T03:04:42.543277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:39:00.494720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.064951+0000","last_change":"2024-09-12T03:03:07.446854+0000","last_active":"2024-09-12T03:04:43.064951+0000","last_peered":"2024-09-12T03:04:43.064951+0000","last_clean":"2024-09-12T03:04:43.064951+0000","last_became_active":"2024-09-12T03:03:07.446411+0000","last_became_peered":"2024-09-12T03:03:07.446411+0000","last_unstale":"2024-09-12T03:04:43.064951+0000","last_undegraded":"2024-09-12T03:04:43.064951+0000","last_fullsized":"2024-09-12T03:04:43.064951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:54.917686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.457958+0000","last_change":"2024-09-12T03:03:07.448332+0000","last_active":"2024-09-12T03:04:42.457958+0000","last_peered":"2024-09-12T03:04:42.457958+0000","last_clean":"2024-09-12T03:04:42.457958+0000","last_became_active":"2024-09-12T03:03:07.447776+0000","last_became_peered":"2024-09-12T03:03:07.447776+0000","last_unstale":"2024-09-12T03:04:42.457958+0000","last_undegraded":"2024-09-12T03:04:42.457958+0000","last_fullsized":"2024-09-12T03:04:42.457958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:08.531088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.458250+0000","last_change":"2024-09-12T03:03:07.449781+0000","last_active":"2024-09-12T03:04:42.458250+0000","last_peered":"2024-09-12T03:04:42.458250+0000","last_clean":"2024-09-12T03:04:42.458250+0000","last_became_active":"2024-09-12T03:03:07.449359+0000","last_became_peered":"2024-09-12T03:03:07.449359+0000","last_unstale":"2024-09-12T03:04:42.458250+0000","last_undegraded":"2024-09-12T03:04:42.458250+0000","last_fullsized":"2024-09-12T03:04:42.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:11.121922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.542793+0000","last_change":"2024-09-12T03:03:07.453881+0000","last_active":"2024-09-12T03:04:42.542793+0000","last_peered":"2024-09-12T03:04:42.542793+0000","last_clean":"2024-09-12T03:04:42.542793+0000","last_became_active":"2024-09-12T03:03:07.453455+0000","last_became_peered":"2024-09-12T03:03:07.453455+0000","last_unstale":"2024-09-12T03:04:42.542793+0000","last_undegraded":"2024-09-12T03:04:42.542793+0000","last_fullsized":"2024-09-12T03:04:42.542793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:18:46.797598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.462235+0000","last_change":"2024-09-12T03:03:07.459332+0000","last_active":"2024-09-12T03:04:42.462235+0000","last_peered":"2024-09-12T03:04:42.462235+0000","last_clean":"2024-09-12T03:04:42.462235+0000","last_became_active":"2024-09-12T03:03:07.459032+0000","last_became_peered":"2024-09-12T03:03:07.459032+0000","last_unstale":"2024-09-12T03:04:42.462235+0000","last_undegraded":"2024-09-12T03:04:42.462235+0000","last_fullsized":"2024-09-12T03:04:42.462235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:16.702782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.851092+0000","last_change":"2024-09-12T03:03:07.441696+0000","last_active":"2024-09-12T03:04:42.851092+0000","last_peered":"2024-09-12T03:04:42.851092+0000","last_clean":"2024-09-12T03:04:42.851092+0000","last_became_active":"2024-09-12T03:03:07.440575+0000","last_became_peered":"2024-09-12T03:03:07.440575+0000","last_unstale":"2024-09-12T03:04:42.851092+0000","last_undegraded":"2024-09-12T03:04:42.851092+0000","last_fullsized":"2024-09-12T03:04:42.851092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:19:26.049893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016765+0000","last_change":"2024-09-12T03:03:07.454945+0000","last_active":"2024-09-12T03:04:43.016765+0000","last_peered":"2024-09-12T03:04:43.016765+0000","last_clean":"2024-09-12T03:04:43.016765+0000","last_became_active":"2024-09-12T03:03:07.452368+0000","last_became_peered":"2024-09-12T03:03:07.452368+0000","last_unstale":"2024-09-12T03:04:43.016765+0000","last_undegraded":"2024-09-12T03:04:43.016765+0000","last_fullsized":"2024-09-12T03:04:43.016765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:21.169859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:43.016920+0000","last_change":"2024-09-12T03:03:07.454230+0000","last_active":"2024-09-12T03:04:43.016920+0000","last_peered":"2024-09-12T03:04:43.016920+0000","last_clean":"2024-09-12T03:04:43.016920+0000","last_became_active":"2024-09-12T03:03:07.453913+0000","last_became_peered":"2024-09-12T03:03:07.453913+0000","last_unstale":"2024-09-12T03:04:43.016920+0000","last_undegraded":"2024-09-12T03:04:43.016920+0000","last_fullsized":"2024-09-12T03:04:43.016920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:56.264048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-12T03:04:42.464571+0000","last_change":"2024-09-12T03:03:07.449227+0000","last_active":"2024-09-12T03:04:42.464571+0000","last_peered":"2024-09-12T03:04:42.464571+0000","last_clean":"2024-09-12T03:04:42.464571+0000","last_became_active":"2024-09-12T03:03:07.448605+0000","last_became_peered":"2024-09-12T03:03:07.448605+0000","last_unstale":"2024-09-12T03:04:42.464571+0000","last_undegraded":"2024-09-12T03:04:42.464571+0000","last_fullsized":"2024-09-12T03:04:42.464571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:03:06.416148+0000","last_clean_scrub_stamp":"2024-09-12T03:03:06.416148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:33:06.420752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66664,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":414,"num_read_kb":277,"num_write":167,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481830,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":554372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710761,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712208,"statfs":{"total":95995035648,"available":95961300992,"internally_reserved":0,"allocated":1798144,"data_stored":1204301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874286,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":558704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1122304,"data_stored":554528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1716224,"data_stored":1150439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":549784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1724416,"data_stored":1146027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":554434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":549978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36516,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708636,"statfs":{"total":95995035648,"available":95957643264,"internally_reserved":0,"allocated":1196032,"data_stored":614098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36496,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708656,"statfs":{"total":95995035648,"available":95957663744,"internally_reserved":0,"allocated":1171456,"data_stored":612355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712860,"statfs":{"total":95995035648,"available":95961968640,"internally_reserved":0,"allocated":1126400,"data_stored":551172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:05:05.214 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:05:05.214 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:05:05.214 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:05:05.214 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:05:05.601 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:05.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:05 smithi057 ceph-mon[30361]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:05.740 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:05 smithi057 ceph-mon[30361]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:05 smithi204 ceph-mon[35013]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:05 smithi204 ceph-mon[35013]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:05.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:05 smithi181 ceph-mon[35320]: pgmap v368: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:05.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:05 smithi181 ceph-mon[35320]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:05:06.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 -- 172.21.15.57:0/2888172807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074f40 msgr2=0x7f1db010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 --2- 172.21.15.57:0/2888172807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074f40 0x7f1db010de80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02a480 comp rx=0 tx=0).stop 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 -- 172.21.15.57:0/2888172807 shutdown_connections 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 --2- 172.21.15.57:0/2888172807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db010e3c0 0x7f1db0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 --2- 172.21.15.57:0/2888172807 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074f40 0x7f1db010de80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 --2- 172.21.15.57:0/2888172807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db0074620 0x7f1db0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 -- 172.21.15.57:0/2888172807 >> 172.21.15.57:0/2888172807 conn(0x7f1db00fd8c0 msgr2=0x7f1db00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 -- 172.21.15.57:0/2888172807 shutdown_connections 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.151+0000 7f1db8400640 1 -- 172.21.15.57:0/2888172807 wait complete. 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 Processor -- start 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 -- start start 2024-09-12T03:05:06.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db0074f40 0x7f1db019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db010e3c0 0x7f1db019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01141b0 con 0x7f1db0074620 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0114030 con 0x7f1db010e3c0 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db8400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0114330 con 0x7f1db0074f40 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db5a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42844/0 (socket says 172.21.15.57:42844) 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.152+0000 7f1db5a00640 1 -- 172.21.15.57:0/260602620 learned_addr learned my addr 172.21.15.57:0/260602620 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5000640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db0074f40 0x7f1db019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db6400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db010e3c0 0x7f1db019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 -- 172.21.15.57:0/260602620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db0074f40 msgr2=0x7f1db019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db0074f40 0x7f1db019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 -- 172.21.15.57:0/260602620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db010e3c0 msgr2=0x7f1db019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:06.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db010e3c0 0x7f1db019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 -- 172.21.15.57:0/260602620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db019f620 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.153+0000 7f1db5a00640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1da400e9f0 tx=0x7f1da400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.154+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da400cd80 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.154+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da4010910 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.154+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db019f910 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.154+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4018990 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.154+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db01a71f0 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.155+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db0074a00 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.157+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1da4010ac0 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.158+0000 7f1dae000640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 0x7f1d980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.158+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1da40cf870 con 0x7f1db0074620 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.158+0000 7f1db5000640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 0x7f1d980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:06.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.159+0000 7f1db5000640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 0x7f1d980a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:06.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.162+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1da409bbf0 con 0x7f1db0074620 2024-09-12T03:05:06.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.391+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1db0109570 con 0x7f1db0074620 2024-09-12T03:05:06.393 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.391+0000 7f1dae000640 1 -- 172.21.15.57:0/260602620 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1da409fef0 con 0x7f1db0074620 2024-09-12T03:05:06.393 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:06.393 INFO:teuthology.orchestra.run.smithi057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:05:06.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 msgr2=0x7f1d980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:06.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 0x7f1d980a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000c00 tx=0x7f1d9c005b50 comp rx=0 tx=0).stop 2024-09-12T03:05:06.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 msgr2=0x7f1db019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:06.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1da400e9f0 tx=0x7f1da400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 shutdown_connections 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1d980a2c90 0x7f1d980a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1db010e3c0 0x7f1db019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db0074f40 0x7f1db019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 --2- 172.21.15.57:0/260602620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1db0074620 0x7f1db019a670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.393+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 >> 172.21.15.57:0/260602620 conn(0x7f1db00fd8c0 msgr2=0x7f1db00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.394+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 shutdown_connections 2024-09-12T03:05:06.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:06.394+0000 7f1db8400640 1 -- 172.21.15.57:0/260602620 wait complete. 2024-09-12T03:05:06.749 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:06 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/260602620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:05:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:06 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/260602620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:05:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:06 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/260602620' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:05:07.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:05:07.194 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:07.573 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:07.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:07 smithi057 ceph-mon[30361]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:07 smithi204 ceph-mon[35013]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:07 smithi181 ceph-mon[35320]: pgmap v369: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:08.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.134+0000 7fee39800640 1 -- 172.21.15.57:0/193495844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341049c0 msgr2=0x7fee34108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.134+0000 7fee39800640 1 --2- 172.21.15.57:0/193495844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341049c0 0x7fee34108a10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 -- 172.21.15.57:0/193495844 shutdown_connections 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 --2- 172.21.15.57:0/193495844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34109140 0x7fee34110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 --2- 172.21.15.57:0/193495844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee341049c0 0x7fee34108a10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 --2- 172.21.15.57:0/193495844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34104010 0x7fee341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 -- 172.21.15.57:0/193495844 >> 172.21.15.57:0/193495844 conn(0x7fee340ffa50 msgr2=0x7fee34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 -- 172.21.15.57:0/193495844 shutdown_connections 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.135+0000 7fee39800640 1 -- 172.21.15.57:0/193495844 wait complete. 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 Processor -- start 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 -- start start 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34104010 0x7fee3419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee341049c0 0x7fee3419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fee34114190 con 0x7fee34109140 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee34114010 con 0x7fee341049c0 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee39800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee34114310 con 0x7fee34104010 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.136+0000 7fee33400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41904/0 (socket says 172.21.15.57:41904) 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 -- 172.21.15.57:0/4231713987 learned_addr learned my addr 172.21.15.57:0/4231713987 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee2be00640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee341049c0 0x7fee3419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee32a00640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34104010 0x7fee3419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 -- 172.21.15.57:0/4231713987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34104010 msgr2=0x7fee3419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34104010 0x7fee3419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 -- 172.21.15.57:0/4231713987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee341049c0 msgr2=0x7fee3419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee341049c0 0x7fee3419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 -- 172.21.15.57:0/4231713987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419f6c0 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee33400640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009fd0 tx=0x7fee2c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.137+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c017070 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.138+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c003e50 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.138+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0106c0 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.138+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419f9b0 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.138+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341a7290 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.139+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee341043f0 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.141+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee2c003970 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.142+0000 7fee2b400640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 0x7fee180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.142+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fee2c0cf2c0 con 0x7fee34109140 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.142+0000 7fee32a00640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 0x7fee180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.143+0000 7fee32a00640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 0x7fee180a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee2400bbe0 tx=0x7fee2400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:08.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.146+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee2c09b640 con 0x7fee34109140 2024-09-12T03:05:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.368+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee3410c380 con 0x7fee34109140 2024-09-12T03:05:08.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.369+0000 7fee2b400640 1 -- 172.21.15.57:0/4231713987 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7fee2c097020 con 0x7fee34109140 2024-09-12T03:05:08.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:08.370 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:05:08.370 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.371+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 msgr2=0x7fee180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.371+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 0x7fee180a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fee2400bbe0 tx=0x7fee2400b390 comp rx=0 tx=0).stop 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.371+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 msgr2=0x7fee3419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.371+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009fd0 tx=0x7fee2c00b260 comp rx=0 tx=0).stop 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.371+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 shutdown_connections 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fee180a2c10 0x7fee180a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee34109140 0x7fee3419efe0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee341049c0 0x7fee3419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 --2- 172.21.15.57:0/4231713987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee34104010 0x7fee3419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 >> 172.21.15.57:0/4231713987 conn(0x7fee340ffa50 msgr2=0x7fee34100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 shutdown_connections 2024-09-12T03:05:08.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:08.372+0000 7fee39800640 1 -- 172.21.15.57:0/4231713987 wait complete. 2024-09-12T03:05:08.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:08 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4231713987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:08 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/4231713987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:08 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/4231713987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:09.243 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:09.623 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:09.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:09 smithi057 ceph-mon[30361]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:09 smithi204 ceph-mon[35013]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:09 smithi181 ceph-mon[35320]: pgmap v370: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:10.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 -- 172.21.15.57:0/1732869174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 msgr2=0x7f05341021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 --2- 172.21.15.57:0/1732869174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f05341021d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0524009900 tx=0x7f052402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 -- 172.21.15.57:0/1732869174 shutdown_connections 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 --2- 172.21.15.57:0/1732869174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 0x7f053410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 --2- 172.21.15.57:0/1732869174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 0x7f05341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 --2- 172.21.15.57:0/1732869174 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f05341021d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.229+0000 7f053ac00640 1 -- 172.21.15.57:0/1732869174 >> 172.21.15.57:0/1732869174 conn(0x7f05340fd8b0 msgr2=0x7f05340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.230+0000 7f053ac00640 1 -- 172.21.15.57:0/1732869174 shutdown_connections 2024-09-12T03:05:10.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.230+0000 7f053ac00640 1 -- 172.21.15.57:0/1732869174 wait complete. 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 Processor -- start 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 -- start start 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 0x7f053419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 0x7f053419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0534111fd0 con 0x7f0534101df0 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0534111e50 con 0x7f05341027a0 2024-09-12T03:05:10.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f053ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0534112150 con 0x7f0534106f20 2024-09-12T03:05:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f0533e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f0533e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41918/0 (socket says 172.21.15.57:41918) 2024-09-12T03:05:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f0533e00640 1 -- 172.21.15.57:0/66358219 learned_addr learned my addr 172.21.15.57:0/66358219 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.231+0000 7f0533400640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 0x7f053419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0538c00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 0x7f053419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 -- 172.21.15.57:0/66358219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 msgr2=0x7f053419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 0x7f053419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 -- 172.21.15.57:0/66358219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 msgr2=0x7f053419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 0x7f053419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 -- 172.21.15.57:0/66358219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053419f560 con 0x7f0534101df0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.232+0000 7f0533e00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0524004770 tx=0x7f0524003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.233+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052403f070 con 0x7f0534101df0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.233+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524031ca0 con 0x7f0534101df0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.233+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053419f7f0 con 0x7f0534101df0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.233+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524035030 con 0x7f0534101df0 2024-09-12T03:05:10.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.233+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05341a7130 con 0x7f0534101df0 2024-09-12T03:05:10.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.234+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05341021d0 con 0x7f0534101df0 2024-09-12T03:05:10.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.236+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0524042070 con 0x7f0534101df0 2024-09-12T03:05:10.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.238+0000 7f0530c00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 0x7f05180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:10.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.238+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f05240eff80 con 0x7f0534101df0 2024-09-12T03:05:10.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.238+0000 7f0533400640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 0x7f05180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:10.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.239+0000 7f0533400640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 0x7f05180a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0534076480 tx=0x7f0520005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:10.240 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.240+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05240bc3c0 con 0x7f0534101df0 2024-09-12T03:05:10.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.460+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05341a7330 con 0x7f0534101df0 2024-09-12T03:05:10.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.461+0000 7f0530c00640 1 -- 172.21.15.57:0/66358219 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7f05240c06c0 con 0x7f0534101df0 2024-09-12T03:05:10.460 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:10.461 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:05:10.461 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 msgr2=0x7f05180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 0x7f05180a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0534076480 tx=0x7f0520005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 msgr2=0x7f053419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0524004770 tx=0x7f0524003c10 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 shutdown_connections 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f05180a2c10 0x7f05180a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0534106f20 0x7f053419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05341027a0 0x7f053419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 --2- 172.21.15.57:0/66358219 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0534101df0 0x7f053419a5b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 >> 172.21.15.57:0/66358219 conn(0x7f05340fd8b0 msgr2=0x7f05340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:10.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 shutdown_connections 2024-09-12T03:05:10.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:10.463+0000 7f053ac00640 1 -- 172.21.15.57:0/66358219 wait complete. 2024-09-12T03:05:10.863 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:10 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:10.864 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:10 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/66358219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:10.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:10 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:10.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:10 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/66358219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:10 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:10 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/66358219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:11.269 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:05:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:11 smithi181 ceph-mon[35320]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:11.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:11 smithi057 ceph-mon[30361]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:11.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:11 smithi204 ceph-mon[35013]: pgmap v371: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:12.297 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 -- 172.21.15.57:0/1013660818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8108f60 msgr2=0x7f22d8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 --2- 172.21.15.57:0/1013660818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8108f60 0x7f22d8110ae0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009be0 tx=0x7f22d002a6c0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 -- 172.21.15.57:0/1013660818 shutdown_connections 2024-09-12T03:05:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 --2- 172.21.15.57:0/1013660818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8108f60 0x7f22d8110ae0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.887 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 --2- 172.21.15.57:0/1013660818 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d81047e0 0x7f22d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 --2- 172.21.15.57:0/1013660818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d8103e30 0x7f22d8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 -- 172.21.15.57:0/1013660818 >> 172.21.15.57:0/1013660818 conn(0x7f22d80ff8d0 msgr2=0x7f22d8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 -- 172.21.15.57:0/1013660818 shutdown_connections 2024-09-12T03:05:12.888 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.887+0000 7f22dca00640 1 -- 172.21.15.57:0/1013660818 wait complete. 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.888+0000 7f22dca00640 1 Processor -- start 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.888+0000 7f22dca00640 1 -- start start 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 0x7f22d819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d8108f60 0x7f22d819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8113fc0 con 0x7f22d8108f60 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8113e40 con 0x7f22d81047e0 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22dca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8114140 con 0x7f22d8103e30 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22d6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 0x7f22d819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22d7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d8108f60 0x7f22d819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:59822/0 (socket says 172.21.15.57:59822) 2024-09-12T03:05:12.889 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22d6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 0x7f22d819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51244/0 (socket says 172.21.15.57:51244) 2024-09-12T03:05:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.889+0000 7f22cfe00640 1 -- 172.21.15.57:0/2584162276 learned_addr learned my addr 172.21.15.57:0/2584162276 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 -- 172.21.15.57:0/2584162276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 msgr2=0x7f22d819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 0x7f22d819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 -- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d8108f60 msgr2=0x7f22d819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:12.890 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d8108f60 0x7f22d819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 -- 172.21.15.57:0/2584162276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d819f3d0 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cfe00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22c000e9c0 tx=0x7f22c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c000ccc0 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d819f6c0 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22c0005500 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.890+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d81a6fa0 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.891+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c0010430 con 0x7f22d81047e0 2024-09-12T03:05:12.891 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.892+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4005290 con 0x7f22d81047e0 2024-09-12T03:05:12.893 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.894+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f22c00046c0 con 0x7f22d81047e0 2024-09-12T03:05:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.895+0000 7f22cf400640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 0x7f22bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.895+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f22c00cf630 con 0x7f22d81047e0 2024-09-12T03:05:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.895+0000 7f22d6800640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 0x7f22bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:12.895 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.896+0000 7f22d6800640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 0x7f22bc0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009bd0 tx=0x7f22c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:12.898 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:12.899+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22c009c000 con 0x7f22d81047e0 2024-09-12T03:05:13.101 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.101+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22b4005850 con 0x7f22d81047e0 2024-09-12T03:05:13.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.103+0000 7f22cf400640 1 -- 172.21.15.57:0/2584162276 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24911 (secure 0 0 0) 0x7f22c009b900 con 0x7f22d81047e0 2024-09-12T03:05:13.103 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:13.104 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":96,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:04:42.450539+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:05:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 msgr2=0x7f22bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:13.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 0x7f22bc0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009bd0 tx=0x7f22c8005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 msgr2=0x7f22d819a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22c000e9c0 tx=0x7f22c000ee80 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 shutdown_connections 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f22bc0a2c10 0x7f22bc0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f22d8108f60 0x7f22d819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d81047e0 0x7f22d819a960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 --2- 172.21.15.57:0/2584162276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d8103e30 0x7f22d819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 >> 172.21.15.57:0/2584162276 conn(0x7f22d80ff8d0 msgr2=0x7f22d8100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 shutdown_connections 2024-09-12T03:05:13.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:13.106+0000 7f22dca00640 1 -- 172.21.15.57:0/2584162276 wait complete. 2024-09-12T03:05:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:13 smithi057 ceph-mon[30361]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:13 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:13 smithi204 ceph-mon[35013]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:13 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:13 smithi181 ceph-mon[35320]: pgmap v372: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:13 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:13.922 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:14.335 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:14.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:14 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2584162276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:14.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:14 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2584162276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:14.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:14 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2584162276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 -- 172.21.15.57:0/702172363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8104010 msgr2=0x7f3df81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 --2- 172.21.15.57:0/702172363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8104010 0x7f3df81043f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009900 tx=0x7f3de802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 -- 172.21.15.57:0/702172363 shutdown_connections 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 --2- 172.21.15.57:0/702172363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df8109140 0x7f3df8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 --2- 172.21.15.57:0/702172363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df81049c0 0x7f3df8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 --2- 172.21.15.57:0/702172363 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8104010 0x7f3df81043f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 -- 172.21.15.57:0/702172363 >> 172.21.15.57:0/702172363 conn(0x7f3df80ffa50 msgr2=0x7f3df8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 -- 172.21.15.57:0/702172363 shutdown_connections 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.922+0000 7f3dff200640 1 -- 172.21.15.57:0/702172363 wait complete. 2024-09-12T03:05:14.941 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 Processor -- start 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 -- start start 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df81049c0 0x7f3df819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 0x7f3df819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8114140 con 0x7f3df8109140 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8113fc0 con 0x7f3df81049c0 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.923+0000 7f3dff200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3df81142c0 con 0x7f3df8104010 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3df7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 0x7f3df819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3df7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51260/0 (socket says 172.21.15.57:51260) 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3df7e00640 1 -- 172.21.15.57:0/5558371 learned_addr learned my addr 172.21.15.57:0/5558371 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:14.942 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3df7400640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df81049c0 0x7f3df819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 -- 172.21.15.57:0/5558371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 msgr2=0x7f3df819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 -- 172.21.15.57:0/5558371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df81049c0 msgr2=0x7f3df819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df81049c0 0x7f3df819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3dfd200640 1 -- 172.21.15.57:0/5558371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df819f5b0 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.924+0000 7f3df7e00640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3dfd200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 0x7f3df819eed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009fd0 tx=0x7f3df000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000ce20 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0005020 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df819f8a0 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0010720 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.925+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df81a7180 con 0x7f3df8109140 2024-09-12T03:05:14.943 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.926+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df81043f0 con 0x7f3df8109140 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.928+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3df0005540 con 0x7f3df8109140 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.930+0000 7f3df4c00640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 0x7f3ddc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.930+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3df00cf7f0 con 0x7f3df8109140 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.930+0000 7f3df7e00640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 0x7f3ddc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.931+0000 7f3df7e00640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 0x7f3ddc0a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3de80096c0 tx=0x7f3de8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:14.944 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:14.934+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3df009c1c0 con 0x7f3df8109140 2024-09-12T03:05:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.158+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3df802d7c0 con 0x7f3df8109140 2024-09-12T03:05:15.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.159+0000 7f3df4c00640 1 -- 172.21.15.57:0/5558371 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7f3df009bac0 con 0x7f3df8109140 2024-09-12T03:05:15.159 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:15.159 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:05:15.159 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:15.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 msgr2=0x7f3ddc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:15.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 0x7f3ddc0a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3de80096c0 tx=0x7f3de8005a20 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 msgr2=0x7f3df819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 0x7f3df819eed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009fd0 tx=0x7f3df000ef10 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 shutdown_connections 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ddc0a2c90 0x7f3ddc0a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3df8109140 0x7f3df819eed0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3df81049c0 0x7f3df819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 --2- 172.21.15.57:0/5558371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3df8104010 0x7f3df819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 >> 172.21.15.57:0/5558371 conn(0x7f3df80ffa50 msgr2=0x7f3df8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 shutdown_connections 2024-09-12T03:05:15.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:15.162+0000 7f3dff200640 1 -- 172.21.15.57:0/5558371 wait complete. 2024-09-12T03:05:15.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:15.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.529 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.530 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:15 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/5558371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:15.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:15.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:15 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/5558371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: pgmap v373: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:05:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='mgr.14150 172.21.15.57:0/1056651349' entity='mgr.x' 2024-09-12T03:05:15.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:15 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/5558371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:15.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:03:08.679612+0000', 'modified': '2024-09-12T03:03:58.893616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24548, 'mds_2': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1804109241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1804109241}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1804109241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/368351857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 368351857}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 368351857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/2458916124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2458916124}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2458916124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.204:6839/3278241191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 3278241191}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 3278241191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.57:6841/1479422251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 1479422251}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 1479422251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6839/399579254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 399579254}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 399579254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24529, 24548, 24554]} 2024-09-12T03:05:15.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:05:15.972 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:05:16.350 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:16.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 -- 172.21.15.57:0/1997795449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 msgr2=0x7f8c341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 --2- 172.21.15.57:0/1997795449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c341067e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009900 tx=0x7f8c1c02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 -- 172.21.15.57:0/1997795449 shutdown_connections 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 --2- 172.21.15.57:0/1997795449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34106f10 0x7f8c3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 --2- 172.21.15.57:0/1997795449 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c341067e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 --2- 172.21.15.57:0/1997795449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34101de0 0x7f8c341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 -- 172.21.15.57:0/1997795449 >> 172.21.15.57:0/1997795449 conn(0x7f8c340fd8c0 msgr2=0x7f8c340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 -- 172.21.15.57:0/1997795449 shutdown_connections 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.955+0000 7f8c3a600640 1 -- 172.21.15.57:0/1997795449 wait complete. 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 Processor -- start 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 -- start start 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34101de0 0x7f8c3419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 0x7f8c3419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34111f70 con 0x7f8c34102790 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34111df0 con 0x7f8c34106f10 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.956+0000 7f8c3a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341120f0 con 0x7f8c34101de0 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41954/0 (socket says 172.21.15.57:41954) 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 -- 172.21.15.57:0/3456927969 learned_addr learned my addr 172.21.15.57:0/3456927969 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c33400640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34101de0 0x7f8c3419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 0x7f8c3419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 -- 172.21.15.57:0/3456927969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34101de0 msgr2=0x7f8c3419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34101de0 0x7f8c3419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 -- 172.21.15.57:0/3456927969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 msgr2=0x7f8c3419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 0x7f8c3419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 -- 172.21.15.57:0/3456927969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3419f5b0 con 0x7f8c34102790 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 0x7f8c3419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.957+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.958+0000 7f8c30c00640 1 -- 172.21.15.57:0/3456927969 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c03f030 con 0x7f8c34102790 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.958+0000 7f8c30c00640 1 -- 172.21.15.57:0/3456927969 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1c004440 con 0x7f8c34102790 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.958+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3419f840 con 0x7f8c34102790 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.958+0000 7f8c30c00640 1 -- 172.21.15.57:0/3456927969 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c035070 con 0x7f8c34102790 2024-09-12T03:05:16.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.958+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a7180 con 0x7f8c34102790 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.961+0000 7f8c30c00640 1 -- 172.21.15.57:0/3456927969 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8c1c042030 con 0x7f8c34102790 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.961+0000 7f8c3d098740 1 Processor -- start 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.961+0000 7f8c3d098740 1 -- start start 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8c50 0x55e90f202fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e90f15e0b0 con 0x55e90f1f8c50 2024-09-12T03:05:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e90f105ab0 con 0x55e90f203510 2024-09-12T03:05:16.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c3d098740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e90f15a070 con 0x55e90f1f8330 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51276/0 (socket says 172.21.15.57:51276) 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:59852/0 (socket says 172.21.15.57:59852) 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c33e00640 1 -- 172.21.15.57:0/3784616782 learned_addr learned my addr 172.21.15.57:0/3784616782 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.962+0000 7f8c33400640 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8c50 0x55e90f202fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 -- 172.21.15.57:0/3784616782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 msgr2=0x55e90f205900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 -- 172.21.15.57:0/3784616782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8c50 msgr2=0x55e90f202fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8c50 0x55e90f202fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 -- 172.21.15.57:0/3784616782 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90f1124a0 con 0x55e90f1f8330 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c30c00640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 0x7f8c180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c30c00640 1 -- 172.21.15.57:0/3456927969 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8c1c0ef950 con 0x7f8c34102790 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c011790 tx=0x7f8c2c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4568c886b3ad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c33400640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 0x7f8c180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.963+0000 7f8c28c00640 1 -- 172.21.15.57:0/3784616782 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c003c30 con 0x55e90f1f8330 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c28c00640 1 -- 172.21.15.57:0/3784616782 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c00fe80 con 0x55e90f1f8330 2024-09-12T03:05:16.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c28c00640 1 -- 172.21.15.57:0/3784616782 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0193e0 con 0x55e90f1f8330 2024-09-12T03:05:16.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c3d098740 1 -- 172.21.15.57:0/3784616782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 msgr2=0x55e90f1f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c011790 tx=0x7f8c2c011c50 comp rx=0 tx=0).stop 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c33400640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 0x7f8c180a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55e90f1f5720 tx=0x7f8c24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c3d098740 1 -- 172.21.15.57:0/3784616782 shutdown_connections 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.964+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f205900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8c50 0x55e90f202fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3784616782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8330 0x55e90f1f8710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 -- 172.21.15.57:0/3784616782 >> 172.21.15.57:0/3784616782 conn(0x55e90f1f48c0 msgr2=0x55e90f1f4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 -- 172.21.15.57:0/3784616782 shutdown_connections 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 -- 172.21.15.57:0/3784616782 wait complete. 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 -- start start 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.965+0000 7f8c3d098740 1 Processor -- start 2024-09-12T03:05:16.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f2005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f200af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e90f206060 con 0x55e90f203510 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e90f0e6920 con 0x55e90f1f8330 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e90f2061e0 con 0x55e90f1f8c50 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c3d098740 10 client.0 _ll_register_callbacks cb 0x55e90f10a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41974/0 (socket says 172.21.15.57:41974) 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c33e00640 1 -- 172.21.15.57:0/3465523217 learned_addr learned my addr 172.21.15.57:0/3465523217 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f200af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.966+0000 7f8c33400640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f2005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 -- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 msgr2=0x55e90f200af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.969 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f200af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 -- 172.21.15.57:0/3465523217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 msgr2=0x55e90f2005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f2005b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 -- 172.21.15.57:0/3465523217 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90eecf2b0 con 0x55e90f203510 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c33e00640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe20 tx=0x7f8c2c004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51deca6cac2f64f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c21600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c01f040 con 0x55e90f203510 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8c2c011340 con 0x55e90f203510 2024-09-12T03:05:16.970 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.967+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e90f16b620 con 0x55e90f203510 2024-09-12T03:05:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c032050 con 0x55e90f203510 2024-09-12T03:05:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 10 client.14949 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e90f211e40 con 0x55e90f203510 2024-09-12T03:05:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e90f16b620 con 0x55e90f203510 2024-09-12T03:05:16.972 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8c2c018070 con 0x55e90f203510 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 10 client.14949 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 10 client.14949 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 20 client.14949 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f214200 0x55e90f215670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 4 client.14949 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.968+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 --> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e90f165b30 con 0x55e90f214200 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.969+0000 7f8c33400640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f214200 0x55e90f215670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.969+0000 7f8c33400640 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f214200 0x55e90f215670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25a4855d11cdcdcb server_cookie=e54d6c30e6c2db8 in_seq=0 out_seq=0 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.969+0000 7f8c21600640 10 client.14949 ms_handle_connect on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.970+0000 7f8c21600640 1 -- 172.21.15.57:0/3465523217 <== mds.2 v2:172.21.15.204:6838/3278241191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e90f165b30 con 0x55e90f214200 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.970+0000 7f8c21600640 10 client.14949 handle_command_reply: tid=0 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.970+0000 7f8c3d098740 1 client.14949 shutdown 2024-09-12T03:05:16.973 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 msgr2=0x55e90f201030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe20 tx=0x7f8c2c004990 comp rx=0 tx=0).stop 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 shutdown_connections 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f214200 0x55e90f215670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f201030 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f200af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 --2- 172.21.15.57:0/3465523217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f2005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 >> 172.21.15.57:0/3465523217 conn(0x55e90f1f48c0 msgr2=0x55e90f1fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c21600640 0 client.14949 ms_handle_reset on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 shutdown_connections 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 1 -- 172.21.15.57:0/3465523217 wait complete. 2024-09-12T03:05:16.974 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.971+0000 7f8c3d098740 20 client.14949 trim_cache size 0 max 16384 2024-09-12T03:05:16.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.989+0000 7f8c3d098740 1 Processor -- start 2024-09-12T03:05:16.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 -- start start 2024-09-12T03:05:16.990 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f200e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f21d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e90f2139b0 con 0x55e90f203510 2024-09-12T03:05:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e90f214080 con 0x55e90f1f8330 2024-09-12T03:05:16.991 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c3d098740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e90f214750 con 0x55e90f1f8c50 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c33400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f200e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f21d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c33400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f200e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:59858/0 (socket says 172.21.15.57:59858) 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c33400640 1 -- 172.21.15.57:0/4261783606 learned_addr learned my addr 172.21.15.57:0/4261783606 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.990+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 -- 172.21.15.57:0/4261783606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 msgr2=0x55e90f1fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 -- 172.21.15.57:0/4261783606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 msgr2=0x55e90f200e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f200e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 -- 172.21.15.57:0/4261783606 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90f1f8a80 con 0x55e90f203510 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c33e00640 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f21d950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe50 tx=0x7f8c2c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34190b125fe86396 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c23e00640 1 -- 172.21.15.57:0/4261783606 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c005370 con 0x55e90f203510 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c23e00640 1 -- 172.21.15.57:0/4261783606 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c029070 con 0x55e90f203510 2024-09-12T03:05:16.993 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.991+0000 7f8c23e00640 1 -- 172.21.15.57:0/4261783606 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c005820 con 0x55e90f203510 2024-09-12T03:05:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 -- 172.21.15.57:0/4261783606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 msgr2=0x55e90f21d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f21d950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe50 tx=0x7f8c2c0044d0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 -- 172.21.15.57:0/4261783606 shutdown_connections 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f203510 0x55e90f21d950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 --2- 172.21.15.57:0/4261783606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f1f8330 0x55e90f200e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 -- 172.21.15.57:0/4261783606 >> 172.21.15.57:0/4261783606 conn(0x55e90f1f48c0 msgr2=0x55e90f1f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 -- 172.21.15.57:0/4261783606 shutdown_connections 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.992+0000 7f8c3d098740 1 -- 172.21.15.57:0/4261783606 wait complete. 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 -- start start 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 Processor -- start 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 0x55e90f21dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f1fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:16.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e90f2154f0 con 0x55e90f1f8330 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e90f214e20 con 0x55e90f203510 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e90f215bc0 con 0x55e90f1f8c50 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c3d098740 10 client.0 _ll_register_callbacks cb 0x7f8c2c00fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.993+0000 7f8c33400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 0x55e90f21dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f1fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c2aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51296/0 (socket says 172.21.15.57:51296) 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c2aa00640 1 -- 172.21.15.57:0/1390399857 learned_addr learned my addr 172.21.15.57:0/1390399857 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 -- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 msgr2=0x55e90f1fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 -- 172.21.15.57:0/1390399857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 msgr2=0x55e90f21dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:16.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 0x55e90f21dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 -- 172.21.15.57:0/1390399857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90f1fe0f0 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33400640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 0x55e90f21dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c2aa00640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c33e00640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f1fd960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe20 tx=0x7f8c2c005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0c392832c8ad04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.994+0000 7f8c21600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c004970 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2c029070 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f8c2c004b10 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c047050 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c3d098740 10 client.25001 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e90f27e910 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.995+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c029070 con 0x55e90f203510 2024-09-12T03:05:16.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8c2c018070 con 0x55e90f203510 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 10 client.25001 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 10 client.25001 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 20 client.25001 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f283170 0x55e90f281e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 4 client.25001 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 --> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] -- command(tid 0: {"prefix": "status"}) -- 0x55e90f0e1f80 con 0x55e90f283170 2024-09-12T03:05:17.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.996+0000 7f8c33400640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f283170 0x55e90f281e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:17.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c33400640 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f283170 0x55e90f281e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9457376c0294acec server_cookie=ae33bf4e55dd3812 in_seq=0 out_seq=0 2024-09-12T03:05:17.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c21600640 10 client.25001 ms_handle_connect on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:17.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c21600640 1 -- 172.21.15.57:0/1390399857 <== mds.2 v2:172.21.15.204:6838/3278241191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e90f0e1f80 con 0x55e90f283170 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c21600640 10 client.25001 handle_command_reply: tid=0 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 client.25001 shutdown 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 msgr2=0x55e90f1fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f1fd960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00fe20 tx=0x7f8c2c005380 comp rx=0 tx=0).stop 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 shutdown_connections 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x55e90f283170 0x55e90f281e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e90f203510 0x55e90f1fd960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e90f1f8c50 0x55e90f1fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c21600640 0 client.25001 ms_handle_reset on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 --2- 172.21.15.57:0/1390399857 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e90f1f8330 0x55e90f21dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 >> 172.21.15.57:0/1390399857 conn(0x55e90f1f48c0 msgr2=0x55e90f20ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 shutdown_connections 2024-09-12T03:05:17.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.997+0000 7f8c3d098740 1 -- 172.21.15.57:0/1390399857 wait complete. 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3d098740 20 client.25001 trim_cache size 0 max 16384 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24529, 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 81.198218527999998, 2024-09-12T03:05:17.003 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 150.921621805 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 msgr2=0x7f8c180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 0x7f8c180a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55e90f1f5720 tx=0x7f8c24005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:17.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 msgr2=0x7f8c3419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c006010 tx=0x7f8c1c004040 comp rx=0 tx=0).stop 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 shutdown_connections 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f8c180a2c10 0x7f8c180a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c34106f10 0x7f8c3419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c34102790 0x7f8c3419ab40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 --2- 172.21.15.57:0/3456927969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c34101de0 0x7f8c3419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.998+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 >> 172.21.15.57:0/3456927969 conn(0x7f8c340fd8c0 msgr2=0x7f8c340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.999+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 shutdown_connections 2024-09-12T03:05:17.005 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:16.999+0000 7f8c3a600640 1 -- 172.21.15.57:0/3456927969 wait complete. 2024-09-12T03:05:17.543 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:17 smithi057 ceph-mon[30361]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:17 smithi204 ceph-mon[35013]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:17 smithi181 ceph-mon[35320]: pgmap v374: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:17.894 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-12T03:05:18.276 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:18.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3226655270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c076720 msgr2=0x7fe99c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3226655270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c076720 0x7fe99c076b00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3226655270 shutdown_connections 2024-09-12T03:05:18.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3226655270 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c112870 0x7fe99c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3226655270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c077040 0x7fe99c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3226655270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c076720 0x7fe99c076b00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.916+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3226655270 >> 172.21.15.57:0/3226655270 conn(0x7fe99c0ff910 msgr2=0x7fe99c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.917+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3226655270 shutdown_connections 2024-09-12T03:05:18.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.917+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3226655270 wait complete. 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.917+0000 7fe9a0c00640 1 Processor -- start 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 -- start start 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 0x7fe99c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 0x7fe99c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c112870 0x7fe99c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c118430 con 0x7fe99c076720 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1182b0 con 0x7fe99c112870 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe9a0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c1185b0 con 0x7fe99c077040 2024-09-12T03:05:18.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe99b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 0x7fe99c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe99b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 0x7fe99c1a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46472/0 (socket says 172.21.15.57:46472) 2024-09-12T03:05:18.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe99b400640 1 -- 172.21.15.57:0/3508636026 learned_addr learned my addr 172.21.15.57:0/3508636026 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:18.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.918+0000 7fe99aa00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 0x7fe99c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99be00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c112870 0x7fe99c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 -- 172.21.15.57:0/3508636026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c112870 msgr2=0x7fe99c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c112870 0x7fe99c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 -- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 msgr2=0x7fe99c1a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 0x7fe99c1a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 -- 172.21.15.57:0/3508636026 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c19ef80 con 0x7fe99c077040 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe99aa00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 0x7fe99c1a4b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9f0 tx=0x7fe98800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe993e00640 1 -- 172.21.15.57:0/3508636026 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800ccc0 con 0x7fe99c077040 2024-09-12T03:05:18.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.919+0000 7fe993e00640 1 -- 172.21.15.57:0/3508636026 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe988010910 con 0x7fe99c077040 2024-09-12T03:05:18.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.920+0000 7fe993e00640 1 -- 172.21.15.57:0/3508636026 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988018800 con 0x7fe99c077040 2024-09-12T03:05:18.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.920+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c19f270 con 0x7fe99c077040 2024-09-12T03:05:18.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.920+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1ab5c0 con 0x7fe99c077040 2024-09-12T03:05:18.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 Processor -- start 2024-09-12T03:05:18.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 -- start start 2024-09-12T03:05:18.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe993e00640 1 -- 172.21.15.57:0/3508636026 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe988010ac0 con 0x7fe99c077040 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b9463648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 0x55b94636f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b946371ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b9462ca0b0 con 0x55b94636f6c0 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b946271ab0 con 0x55b946364e00 2024-09-12T03:05:18.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9462c6070 con 0x55b9463644e0 2024-09-12T03:05:18.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.923+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b946371ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 0x55b94636f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b9463648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 0x55b94636f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54588/0 (socket says 172.21.15.57:54588) 2024-09-12T03:05:18.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99b400640 1 -- 172.21.15.57:0/4395584 learned_addr learned my addr 172.21.15.57:0/4395584 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:18.926 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 -- 172.21.15.57:0/4395584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 msgr2=0x55b94636f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 0x55b94636f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 -- 172.21.15.57:0/4395584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 msgr2=0x55b946371ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b946371ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 -- 172.21.15.57:0/4395584 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94627e4a0 con 0x55b9463644e0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99aa00640 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b946371ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.924+0000 7fe99be00640 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b9463648c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe99c07aa10 tx=0x7fe994011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93583844bc4df5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe993e00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 0x7fe9800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe987e00640 1 -- 172.21.15.57:0/4395584 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994004d20 con 0x55b9463644e0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe987e00640 1 -- 172.21.15.57:0/4395584 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994004050 con 0x55b9463644e0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe987e00640 1 -- 172.21.15.57:0/4395584 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994005280 con 0x55b9463644e0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe993e00640 1 -- 172.21.15.57:0/3508636026 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe9880cf840 con 0x7fe99c077040 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe99b400640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 0x7fe9800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.927 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe9a4c81740 1 -- 172.21.15.57:0/4395584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 msgr2=0x55b9463648c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.925+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b9463648c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe99c07aa10 tx=0x7fe994011810 comp rx=0 tx=0).stop 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 -- 172.21.15.57:0/4395584 shutdown_connections 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b946371ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b946364e00 0x55b94636f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/4395584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b9463648c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 -- 172.21.15.57:0/4395584 >> 172.21.15.57:0/4395584 conn(0x55b946360a70 msgr2=0x55b946360e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 -- 172.21.15.57:0/4395584 shutdown_connections 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe99b400640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 0x7fe9800a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0096c0 tx=0x7fe98c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 -- 172.21.15.57:0/4395584 wait complete. 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.926+0000 7fe9a4c81740 1 -- start start 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 Processor -- start 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 0x55b94636c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b946372330 con 0x55b9463644e0 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b946252920 con 0x55b94636f6c0 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b9463724b0 con 0x55b946364e00 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.927+0000 7fe9a4c81740 10 client.0 _ll_register_callbacks cb 0x55b9462768a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 0x55b94636c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.931 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54590/0 (socket says 172.21.15.57:54590) 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99be00640 1 -- 172.21.15.57:0/184857168 learned_addr learned my addr 172.21.15.57:0/184857168 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 0x55b94636c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46486/0 (socket says 172.21.15.57:46486) 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 -- 172.21.15.57:0/184857168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 msgr2=0x55b94636d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 -- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 msgr2=0x55b94636c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 0x55b94636c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 -- 172.21.15.57:0/184857168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94603b2b0 con 0x55b946364e00 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99be00640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe99aa00640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636cb40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9880ccdc0 tx=0x7fe988097e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f356fed75b91bc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.928+0000 7fe984c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98809b160 con 0x55b946364e00 2024-09-12T03:05:18.932 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe994004050 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe988010430 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98809c210 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 10 client.25018 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b94637e080 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994004050 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe98809a9d0 con 0x55b946364e00 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 10 client.25018 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 10 client.25018 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:05:18.933 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 20 client.25018 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463805f0 0x55b946381a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 4 client.25018 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.929+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 --> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9462d7620 con 0x55b9463805f0 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.930+0000 7fe99b400640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463805f0 0x55b946381a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.930+0000 7fe99b400640 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463805f0 0x55b946381a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d404286b69f6437 server_cookie=f750e4846255ab9f in_seq=0 out_seq=0 2024-09-12T03:05:18.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.930+0000 7fe984c00640 10 client.25018 ms_handle_connect on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.931+0000 7fe984c00640 1 -- 172.21.15.57:0/184857168 <== mds.1 v2:172.21.15.57:6840/1479422251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9462d7620 con 0x55b9463805f0 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.931+0000 7fe984c00640 10 client.25018 handle_command_reply: tid=0 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.931+0000 7fe9a4c81740 1 client.25018 shutdown 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 msgr2=0x55b94636cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636cb40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe9880ccdc0 tx=0x7fe988097e70 comp rx=0 tx=0).stop 2024-09-12T03:05:18.935 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 shutdown_connections 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463805f0 0x55b946381a60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b94636d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636cb40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/184857168 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b9463644e0 0x55b94636c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 >> 172.21.15.57:0/184857168 conn(0x55b946360a70 msgr2=0x55b946366750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe984c00640 0 client.25018 ms_handle_reset on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 shutdown_connections 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 1 -- 172.21.15.57:0/184857168 wait complete. 2024-09-12T03:05:18.936 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.932+0000 7fe9a4c81740 20 client.25018 trim_cache size 0 max 16384 2024-09-12T03:05:18.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.951+0000 7fe9a4c81740 1 Processor -- start 2024-09-12T03:05:18.952 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.951+0000 7fe9a4c81740 1 -- start start 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.951+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9463644e0 0x55b946380cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.951+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b94637fda0 con 0x55b94636f6c0 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b946380470 con 0x55b9463644e0 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b946380b40 con 0x55b946364e00 2024-09-12T03:05:18.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46494/0 (socket says 172.21.15.57:46494) 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 -- 172.21.15.57:0/3072293539 learned_addr learned my addr 172.21.15.57:0/3072293539 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99aa00640 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99b400640 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9463644e0 0x55b946380cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 -- 172.21.15.57:0/3072293539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 msgr2=0x55b94636ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 -- 172.21.15.57:0/3072293539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9463644e0 msgr2=0x55b946380cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9463644e0 0x55b946380cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.952+0000 7fe99be00640 1 -- 172.21.15.57:0/3072293539 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9463608a0 con 0x55b94636f6c0 2024-09-12T03:05:18.958 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.953+0000 7fe99be00640 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9940099a0 tx=0x7fe994009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fa92affb8142d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.953+0000 7fe987400640 1 -- 172.21.15.57:0/3072293539 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994028020 con 0x55b94636f6c0 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.953+0000 7fe987400640 1 -- 172.21.15.57:0/3072293539 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994004050 con 0x55b94636f6c0 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.953+0000 7fe987400640 1 -- 172.21.15.57:0/3072293539 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99401d040 con 0x55b94636f6c0 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- 172.21.15.57:0/3072293539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 msgr2=0x55b94636aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9940099a0 tx=0x7fe994009720 comp rx=0 tx=0).stop 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- 172.21.15.57:0/3072293539 shutdown_connections 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b94636f6c0 0x55b94636aed0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b946364e00 0x55b94636ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/3072293539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b9463644e0 0x55b946380cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- 172.21.15.57:0/3072293539 >> 172.21.15.57:0/3072293539 conn(0x55b946360a70 msgr2=0x55b94635b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- 172.21.15.57:0/3072293539 shutdown_connections 2024-09-12T03:05:18.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- 172.21.15.57:0/3072293539 wait complete. 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.954+0000 7fe9a4c81740 1 -- start start 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 Processor -- start 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b946389ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b9463831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b9463818e0 con 0x55b946364e00 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b946381210 con 0x55b94636f6c0 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b946381fb0 con 0x55b9463644e0 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe9a4c81740 10 client.0 _ll_register_callbacks cb 0x7fe988010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.955+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46502/0 (socket says 172.21.15.57:46502) 2024-09-12T03:05:18.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 -- 172.21.15.57:0/486946436 learned_addr learned my addr 172.21.15.57:0/486946436 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99b400640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b946389ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99be00640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b9463831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 -- 172.21.15.57:0/486946436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 msgr2=0x55b946389ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b946389ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 -- 172.21.15.57:0/486946436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 msgr2=0x55b9463831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b9463831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 -- 172.21.15.57:0/486946436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b946383a00 con 0x55b946364e00 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99b400640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b946389ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe99aa00640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe98809baa0 tx=0x7fe98809b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a30b1083cd6dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe984c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.956+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880b8020 con 0x55b946364e00 2024-09-12T03:05:18.961 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe988097860 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe994004050 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe98809bb10 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 10 client.14973 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b946388480 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994004050 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe9880ba2b0 con 0x55b946364e00 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 10 client.14973 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 10 client.14973 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:05:18.962 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 20 client.14973 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463ef3c0 0x55b9463ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 4 client.14973 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.957+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 --> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] -- command(tid 0: {"prefix": "status"}) -- 0x55b9462d1b30 con 0x55b9463ef3c0 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.958+0000 7fe99b400640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463ef3c0 0x55b9463ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.958+0000 7fe99b400640 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463ef3c0 0x55b9463ee0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b83c9a424493dfe server_cookie=d6a3fffaf768dc36 in_seq=0 out_seq=0 2024-09-12T03:05:18.963 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.958+0000 7fe984c00640 10 client.14973 ms_handle_connect on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.958+0000 7fe984c00640 1 -- 172.21.15.57:0/486946436 <== mds.1 v2:172.21.15.57:6840/1479422251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b9462d1b30 con 0x55b9463ef3c0 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.958+0000 7fe984c00640 10 client.14973 handle_command_reply: tid=0 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 client.14973 shutdown 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 msgr2=0x55b94638a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe98809baa0 tx=0x7fe98809b190 comp rx=0 tx=0).stop 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 shutdown_connections 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55b9463ef3c0 0x55b9463ee0d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.964 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b94636f6c0 0x55b9463831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b946364e00 0x55b94638a3e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 --2- 172.21.15.57:0/486946436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b9463644e0 0x55b946389ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 >> 172.21.15.57:0/486946436 conn(0x55b946360a70 msgr2=0x55b94635f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe984c00640 0 client.14973 ms_handle_reset on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 shutdown_connections 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 1 -- 172.21.15.57:0/486946436 wait complete. 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.959+0000 7fe9a4c81740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24548, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 84.173673475000001, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 144.699641765 2024-09-12T03:05:18.966 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 msgr2=0x7fe9800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 0x7fe9800a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0096c0 tx=0x7fe98c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 msgr2=0x7fe99c1a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 0x7fe99c1a4b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe98800e9f0 tx=0x7fe98800edf0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 shutdown_connections 2024-09-12T03:05:18.967 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe9800a2c90 0x7fe9800a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99c112870 0x7fe99c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe99c077040 0x7fe99c1a4b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 --2- 172.21.15.57:0/3508636026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe99c076720 0x7fe99c1a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 >> 172.21.15.57:0/3508636026 conn(0x7fe99c0ff910 msgr2=0x7fe99c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 shutdown_connections 2024-09-12T03:05:18.968 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:18.960+0000 7fe9a0c00640 1 -- 172.21.15.57:0/3508636026 wait complete. 2024-09-12T03:05:19.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:19 smithi057 ceph-mon[30361]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:19 smithi204 ceph-mon[35013]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:19 smithi181 ceph-mon[35320]: pgmap v375: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:19.902 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:05:20.279 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.824+0000 7f11ed800640 1 -- 172.21.15.57:0/1386063086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 msgr2=0x7f11e81124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.824+0000 7f11ed800640 1 --2- 172.21.15.57:0/1386063086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81124f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009900 tx=0x7f11d402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 -- 172.21.15.57:0/1386063086 shutdown_connections 2024-09-12T03:05:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 --2- 172.21.15.57:0/1386063086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 0x7f11e8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.826 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 --2- 172.21.15.57:0/1386063086 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81124f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 --2- 172.21.15.57:0/1386063086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 -- 172.21.15.57:0/1386063086 >> 172.21.15.57:0/1386063086 conn(0x7f11e80ffa90 msgr2=0x7f11e8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 -- 172.21.15.57:0/1386063086 shutdown_connections 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.825+0000 7f11ed800640 1 -- 172.21.15.57:0/1386063086 wait complete. 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 Processor -- start 2024-09-12T03:05:20.827 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 -- start start 2024-09-12T03:05:20.828 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 0x7f11e819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8118600 con 0x7f11e8077040 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8118480 con 0x7f11e8076720 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11ed800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8118780 con 0x7f11e8112a30 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54628/0 (socket says 172.21.15.57:54628) 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.826+0000 7f11e6a00640 1 -- 172.21.15.57:0/304237129 learned_addr learned my addr 172.21.15.57:0/304237129 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6000640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e7400640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 0x7f11e819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 -- 172.21.15.57:0/304237129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 msgr2=0x7f11e819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 0x7f11e819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 -- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 msgr2=0x7f11e81a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 -- 172.21.15.57:0/304237129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819efc0 con 0x7f11e8076720 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.827+0000 7f11e6a00640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.828+0000 7f11df400640 1 -- 172.21.15.57:0/304237129 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800cd80 con 0x7f11e8076720 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.828+0000 7f11df400640 1 -- 172.21.15.57:0/304237129 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d8010910 con 0x7f11e8076720 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.828+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819f2b0 con 0x7f11e8076720 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.828+0000 7f11df400640 1 -- 172.21.15.57:0/304237129 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8018990 con 0x7f11e8076720 2024-09-12T03:05:20.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.828+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81ab5a0 con 0x7f11e8076720 2024-09-12T03:05:20.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.831+0000 7f11f027f740 1 Processor -- start 2024-09-12T03:05:20.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.831+0000 7f11df400640 1 -- 172.21.15.57:0/304237129 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f11d8010ac0 con 0x7f11e8076720 2024-09-12T03:05:20.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.831+0000 7f11f027f740 1 -- start start 2024-09-12T03:05:20.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 0x562226557fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 0x56222655a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5622264b30b0 con 0x56222654dc50 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56222645aab0 con 0x562226558510 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11f027f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5622264af070 con 0x56222654d330 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 0x562226557fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 0x56222655a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 0x562226557fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46522/0 (socket says 172.21.15.57:46522) 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e6a00640 1 -- 172.21.15.57:0/1866790960 learned_addr learned my addr 172.21.15.57:0/1866790960 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:20.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.832+0000 7f11e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:41178/0 (socket says 172.21.15.57:41178) 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 -- 172.21.15.57:0/1866790960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 msgr2=0x56222655a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 0x56222655a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 -- 172.21.15.57:0/1866790960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 msgr2=0x562226557fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 0x562226557fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 -- 172.21.15.57:0/1866790960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622264674a0 con 0x56222654d330 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e6000640 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 0x56222655a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11e7400640 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11e807aa10 tx=0x7f11e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0b956314cd933b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11df400640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 0x7f11cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11d3e00640 1 -- 172.21.15.57:0/1866790960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0004d20 con 0x56222654d330 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11d3e00640 1 -- 172.21.15.57:0/1866790960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e0004050 con 0x56222654d330 2024-09-12T03:05:20.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11df400640 1 -- 172.21.15.57:0/304237129 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f11d80cf870 con 0x7f11e8076720 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.833+0000 7f11d3e00640 1 -- 172.21.15.57:0/1866790960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0005280 con 0x56222654d330 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11e6a00640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 0x7f11cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 -- 172.21.15.57:0/1866790960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 msgr2=0x56222654d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f11e807aa10 tx=0x7f11e0011810 comp rx=0 tx=0).stop 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11e6a00640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 0x7f11cc0a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11d80a4280 tx=0x7f11d80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 -- 172.21.15.57:0/1866790960 shutdown_connections 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562226558510 0x56222655a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56222654dc50 0x562226557fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 --2- 172.21.15.57:0/1866790960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222654d710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 -- 172.21.15.57:0/1866790960 >> 172.21.15.57:0/1866790960 conn(0x5622265498c0 msgr2=0x562226549ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 -- 172.21.15.57:0/1866790960 shutdown_connections 2024-09-12T03:05:20.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.834+0000 7f11f027f740 1 -- 172.21.15.57:0/1866790960 wait complete. 2024-09-12T03:05:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.835+0000 7f11f027f740 1 -- start start 2024-09-12T03:05:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.835+0000 7f11f027f740 1 Processor -- start 2024-09-12T03:05:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.835+0000 7f11f027f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x5622265555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.835+0000 7f11f027f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.838 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11f027f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226556070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11f027f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56222655b060 con 0x562226558510 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11f027f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56222643b920 con 0x56222654d330 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11f027f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56222655b1e0 con 0x56222654dc50 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11f027f740 10 client.0 _ll_register_callbacks cb 0x56222645f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x5622265555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:41186/0 (socket says 172.21.15.57:41186) 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 -- 172.21.15.57:0/2198054257 learned_addr learned my addr 172.21.15.57:0/2198054257 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e7400640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226556070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 -- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 msgr2=0x5622265555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x5622265555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 -- 172.21.15.57:0/2198054257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 msgr2=0x562226556070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226556070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e6000640 1 -- 172.21.15.57:0/2198054257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622262242b0 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.836+0000 7f11e7400640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226556070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11e6000640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009230 tx=0x7f11d40042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98212ac5f27c16d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4035040 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e0004050 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f11d4005450 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11d404f050 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11f027f740 10 client.25036 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.837+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562226566ed0 con 0x56222654dc50 2024-09-12T03:05:20.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e0004050 con 0x56222654dc50 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f11d40037c0 con 0x56222654dc50 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 10 client.25036 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 10 client.25036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 20 client.25036 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265694c0 0x56222656a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 4 client.25036 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.838+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622264c0620 con 0x5622265694c0 2024-09-12T03:05:20.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.839+0000 7f11e7400640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265694c0 0x56222656a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.839+0000 7f11e7400640 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265694c0 0x56222656a930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfec1250079ff2f3 server_cookie=94bcaa3809a6404c in_seq=0 out_seq=0 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.839+0000 7f11d0c00640 10 client.25036 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11d0c00640 1 -- 172.21.15.57:0/2198054257 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622264c0620 con 0x5622265694c0 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11d0c00640 10 client.25036 handle_command_reply: tid=0 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 client.25036 shutdown 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 msgr2=0x562226555b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009230 tx=0x7f11d40042e0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 shutdown_connections 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265694c0 0x56222656a930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226556070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226555b30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 --2- 172.21.15.57:0/2198054257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x5622265555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 >> 172.21.15.57:0/2198054257 conn(0x5622265498c0 msgr2=0x56222654f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.840+0000 7f11d0c00640 0 client.25036 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:20.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.841+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 shutdown_connections 2024-09-12T03:05:20.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.841+0000 7f11f027f740 1 -- 172.21.15.57:0/2198054257 wait complete. 2024-09-12T03:05:20.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.841+0000 7f11f027f740 20 client.25036 trim_cache size 0 max 16384 2024-09-12T03:05:20.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.859+0000 7f11f027f740 1 Processor -- start 2024-09-12T03:05:20.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.859+0000 7f11f027f740 1 -- start start 2024-09-12T03:05:20.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222656a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 0x562226552890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.860 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562226568c70 con 0x562226558510 2024-09-12T03:05:20.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562226569340 con 0x56222654dc50 2024-09-12T03:05:20.861 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11f027f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562226569a10 con 0x56222654d330 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222656a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 0x562226552890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46534/0 (socket says 172.21.15.57:46534) 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 -- 172.21.15.57:0/3331689306 learned_addr learned my addr 172.21.15.57:0/3331689306 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 -- 172.21.15.57:0/3331689306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 msgr2=0x56222656a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222656a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 -- 172.21.15.57:0/3331689306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 msgr2=0x562226552890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.860+0000 7f11e7400640 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 0x562226552890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11e7400640 1 -- 172.21.15.57:0/3331689306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56222654da40 con 0x562226558510 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11e6000640 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 0x562226552890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:20.862 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11e7400640 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11e0003fd0 tx=0x7f11e0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6b0804880e6164 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11d3400640 1 -- 172.21.15.57:0/3331689306 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e001d040 con 0x562226558510 2024-09-12T03:05:20.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11d3400640 1 -- 172.21.15.57:0/3331689306 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e0004050 con 0x562226558510 2024-09-12T03:05:20.863 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.861+0000 7f11d3400640 1 -- 172.21.15.57:0/3331689306 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0005150 con 0x562226558510 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- 172.21.15.57:0/3331689306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 msgr2=0x562226552dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f11e0003fd0 tx=0x7f11e0009c80 comp rx=0 tx=0).stop 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- 172.21.15.57:0/3331689306 shutdown_connections 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x562226552dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654dc50 0x562226552890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.865 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 --2- 172.21.15.57:0/3331689306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654d330 0x56222656a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- 172.21.15.57:0/3331689306 >> 172.21.15.57:0/3331689306 conn(0x5622265498c0 msgr2=0x562226553ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- 172.21.15.57:0/3331689306 shutdown_connections 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- 172.21.15.57:0/3331689306 wait complete. 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.862+0000 7f11f027f740 1 -- start start 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 Processor -- start 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x562226572d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226573240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x56222656dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56222656a7b0 con 0x562226558510 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56222656a0e0 con 0x56222654d330 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56222656ae80 con 0x56222654dc50 2024-09-12T03:05:20.866 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11f027f740 10 client.0 _ll_register_callbacks cb 0x7f11d4005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11e7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x56222656dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x562226572d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11e6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226573240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x562226572d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54652/0 (socket says 172.21.15.57:54652) 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.863+0000 7f11e7400640 1 -- 172.21.15.57:0/386110724 learned_addr learned my addr 172.21.15.57:0/386110724 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 -- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 msgr2=0x562226572d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x562226572d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 -- 172.21.15.57:0/386110724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 msgr2=0x56222656dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x56222656dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 -- 172.21.15.57:0/386110724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56222656e300 con 0x56222654dc50 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11e6000640 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226573240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009900 tx=0x7f11d4003fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe41990d104bfd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:20.867 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4057070 con 0x56222654dc50 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e0004050 con 0x56222654dc50 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.864+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f11d4005250 con 0x56222654dc50 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11d4060050 con 0x56222654dc50 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 10 client.25042 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622265d4670 con 0x56222654dc50 2024-09-12T03:05:20.868 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11e0004050 con 0x56222654dc50 2024-09-12T03:05:20.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f11d405a020 con 0x56222654dc50 2024-09-12T03:05:20.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 10 client.25042 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:20.869 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 10 client.25042 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 20 client.25042 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265d9160 0x5622265d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 4 client.25042 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.865+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "status"}) -- 0x5622264bab30 con 0x5622265d9160 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11e7400640 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265d9160 0x5622265d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11e7400640 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265d9160 0x5622265d7e70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711510dbf7b92fd5 server_cookie=8737b4ad979158e5 in_seq=0 out_seq=0 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11d0c00640 10 client.25042 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11d0c00640 1 -- 172.21.15.57:0/386110724 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5622264bab30 con 0x5622265d9160 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11d0c00640 10 client.25042 handle_command_reply: tid=0 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11f027f740 1 client.25042 shutdown 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 msgr2=0x562226573240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226573240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009900 tx=0x7f11d4003fe0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.870 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.866+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 shutdown_connections 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5622265d9160 0x5622265d7e70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562226558510 0x56222656dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56222654dc50 0x562226573240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 --2- 172.21.15.57:0/386110724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56222654d330 0x562226572d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 >> 172.21.15.57:0/386110724 conn(0x5622265498c0 msgr2=0x562226563c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11d0c00640 0 client.25042 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 shutdown_connections 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 1 -- 172.21.15.57:0/386110724 wait complete. 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11f027f740 20 client.25042 trim_cache size 0 max 16384 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:20.871 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24554, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 131.415640422, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 144.01670101299999 2024-09-12T03:05:20.872 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 msgr2=0x7f11cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 0x7f11cc0a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11d80a4280 tx=0x7f11d80d5530 comp rx=0 tx=0).stop 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 msgr2=0x7f11e81a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.867+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 shutdown_connections 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f11cc0a2c90 0x7f11cc0a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8112a30 0x7f11e819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.873 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e8077040 0x7f11e81a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 --2- 172.21.15.57:0/304237129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8076720 0x7f11e81a46d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 >> 172.21.15.57:0/304237129 conn(0x7f11e80ffa90 msgr2=0x7f11e8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 shutdown_connections 2024-09-12T03:05:20.874 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:20.868+0000 7f11ed800640 1 -- 172.21.15.57:0/304237129 wait complete. 2024-09-12T03:05:21.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:21 smithi057 ceph-mon[30361]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:21 smithi204 ceph-mon[35013]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:21 smithi181 ceph-mon[35320]: pgmap v376: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:21.856 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:05:21.856 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:05:21.866 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:05:21.867 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:05:22.244 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:22.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.784+0000 7f66da800640 1 -- 172.21.15.57:0/248735228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4105920 msgr2=0x7f66d410c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.784+0000 7f66da800640 1 --2- 172.21.15.57:0/248735228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4105920 0x7f66d410c240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009ca0 tx=0x7f66cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 -- 172.21.15.57:0/248735228 shutdown_connections 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 --2- 172.21.15.57:0/248735228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4105920 0x7f66d410c240 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 --2- 172.21.15.57:0/248735228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4104f60 0x7f66d41053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 --2- 172.21.15.57:0/248735228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d4104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 -- 172.21.15.57:0/248735228 >> 172.21.15.57:0/248735228 conn(0x7f66d40ff5b0 msgr2=0x7f66d41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 -- 172.21.15.57:0/248735228 shutdown_connections 2024-09-12T03:05:22.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.785+0000 7f66da800640 1 -- 172.21.15.57:0/248735228 wait complete. 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 Processor -- start 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 -- start start 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 0x7f66d419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4105920 0x7f66d41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410fab0 con 0x7f66d4104f60 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410f930 con 0x7f66d4103e00 2024-09-12T03:05:22.787 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.786+0000 7f66da800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66d410fc30 con 0x7f66d4105920 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66cbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 0x7f66d419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54680/0 (socket says 172.21.15.57:54680) 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 -- 172.21.15.57:0/1155650778 learned_addr learned my addr 172.21.15.57:0/1155650778 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3e00640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4105920 0x7f66d41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 -- 172.21.15.57:0/1155650778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4105920 msgr2=0x7f66d41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4105920 0x7f66d41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 -- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 msgr2=0x7f66d419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 0x7f66d419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66d3400640 1 -- 172.21.15.57:0/1155650778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d41a1cb0 con 0x7f66d4103e00 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.787+0000 7f66cbe00640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 0x7f66d419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66d3400640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66c400d950 tx=0x7f66c400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:22.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4017030 con 0x7f66d4103e00 2024-09-12T03:05:22.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d41a1fa0 con 0x7f66d4103e00 2024-09-12T03:05:22.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66c4004020 con 0x7f66d4103e00 2024-09-12T03:05:22.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c40047a0 con 0x7f66d4103e00 2024-09-12T03:05:22.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.788+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d41a2450 con 0x7f66d4103e00 2024-09-12T03:05:22.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.789+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d410b120 con 0x7f66d4103e00 2024-09-12T03:05:22.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.791+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f66c40041d0 con 0x7f66d4103e00 2024-09-12T03:05:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.792+0000 7f66d0c00640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 0x7f66b80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.793+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f66c40cf160 con 0x7f66d4103e00 2024-09-12T03:05:22.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.793+0000 7f66cbe00640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 0x7f66b80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:22.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.793+0000 7f66cbe00640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 0x7f66b80a5180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66d419b5a0 tx=0x7f66bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:22.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.795+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66c409bbb0 con 0x7f66d4103e00 2024-09-12T03:05:22.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.997+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f66d4062630 con 0x7f66b80a2cc0 2024-09-12T03:05:22.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:22.998+0000 7f66d0c00640 1 -- 172.21.15.57:0/1155650778 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f66d4062630 con 0x7f66b80a2cc0 2024-09-12T03:05:22.999 INFO:teuthology.orchestra.run.smithi057.stdout:Backend: cephadm 2024-09-12T03:05:22.999 INFO:teuthology.orchestra.run.smithi057.stdout:Available: Yes 2024-09-12T03:05:22.999 INFO:teuthology.orchestra.run.smithi057.stdout:Paused: No 2024-09-12T03:05:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.000+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 msgr2=0x7f66b80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.000+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 0x7f66b80a5180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66d419b5a0 tx=0x7f66bc005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.000+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 msgr2=0x7f66d419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.000+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f66c400d950 tx=0x7f66c400de10 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 shutdown_connections 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f66b80a2cc0 0x7f66b80a5180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66d4105920 0x7f66d41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f66d4104f60 0x7f66d419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 --2- 172.21.15.57:0/1155650778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66d4103e00 0x7f66d419a080 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 >> 172.21.15.57:0/1155650778 conn(0x7f66d40ff5b0 msgr2=0x7f66d41082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 shutdown_connections 2024-09-12T03:05:23.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:23.001+0000 7f66da800640 1 -- 172.21.15.57:0/1155650778 wait complete. 2024-09-12T03:05:23.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:23 smithi057 ceph-mon[30361]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:23 smithi204 ceph-mon[35013]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:23 smithi181 ceph-mon[35320]: pgmap v377: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:23.862 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:05:24.239 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:24.557 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:24 smithi057 ceph-mon[30361]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:24.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:24 smithi204 ceph-mon[35013]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:24.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:24 smithi181 ceph-mon[35320]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 -- 172.21.15.57:0/3531382156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 msgr2=0x7f9020104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 --2- 172.21.15.57:0/3531382156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f9020104200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402a410 comp rx=0 tx=0).stop 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 -- 172.21.15.57:0/3531382156 shutdown_connections 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 --2- 172.21.15.57:0/3531382156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020105920 0x7f902010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 --2- 172.21.15.57:0/3531382156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020104f60 0x7f90201053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 --2- 172.21.15.57:0/3531382156 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f9020104200 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 -- 172.21.15.57:0/3531382156 >> 172.21.15.57:0/3531382156 conn(0x7f90200ff5b0 msgr2=0x7f90201019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:24.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 -- 172.21.15.57:0/3531382156 shutdown_connections 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.810+0000 7f9028800640 1 -- 172.21.15.57:0/3531382156 wait complete. 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 Processor -- start 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 -- start start 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020104f60 0x7f902019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020105920 0x7f90201a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f902010fae0 con 0x7f9020103e00 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f902010f960 con 0x7f9020105920 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.811+0000 7f9028800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f902010fc60 con 0x7f9020104f60 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46576/0 (socket says 172.21.15.57:46576) 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 -- 172.21.15.57:0/3149311727 learned_addr learned my addr 172.21.15.57:0/3149311727 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:24.830 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025400640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020104f60 0x7f902019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9026800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020105920 0x7f90201a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 -- 172.21.15.57:0/3149311727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020104f60 msgr2=0x7f902019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020104f60 0x7f902019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 -- 172.21.15.57:0/3149311727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020105920 msgr2=0x7f90201a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020105920 0x7f90201a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 -- 172.21.15.57:0/3149311727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90201a1cd0 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.812+0000 7f9025e00640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9014031440 tx=0x7f9014032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.813+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901403f030 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.813+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9014003ce0 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.813+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90201a1f60 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.813+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014035070 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.813+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90201a23b0 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.814+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902010b1a0 con 0x7f9020103e00 2024-09-12T03:05:24.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.816+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9014042030 con 0x7f9020103e00 2024-09-12T03:05:24.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.817+0000 7f901aa00640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 0x7f90080a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:24.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.817+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f90140efa90 con 0x7f9020103e00 2024-09-12T03:05:24.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.817+0000 7f9025400640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 0x7f90080a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:24.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.818+0000 7f9025400640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 0x7f90080a5180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f902019b5c0 tx=0x7f9010005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:24.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:24.821+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90140bc460 con 0x7f9020103e00 2024-09-12T03:05:25.020 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.019+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9020062630 con 0x7f90080a2cc0 2024-09-12T03:05:25.032 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.032+0000 7f901aa00640 1 -- 172.21.15.57:0/3149311727 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f9020062630 con 0x7f90080a2cc0 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.a smithi057 running (2m) 10s ago 2m 17.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 b7e872f53d0d 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.b smithi181 running (2m) 12s ago 2m 26.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 e2f6e23d0db7 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.c smithi204 running (2m) 12s ago 2m 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 4a24be7566a3 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.d smithi057 running (2m) 10s ago 2m 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 f92c6bc0fbc0 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.e smithi181 running (2m) 12s ago 2m 14.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 bdef0c4ed621 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.f smithi204 running (2m) 12s ago 2m 19.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 0ece4f5feaf1 2024-09-12T03:05:25.033 INFO:teuthology.orchestra.run.smithi057.stdout:mds.g smithi057 running (2m) 10s ago 2m 12.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 aa03573d3f41 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.h smithi181 running (2m) 12s ago 2m 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 d00e9971ca12 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.i smithi204 running (2m) 12s ago 2m 21.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 2bb60bfc5683 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.j smithi057 running (2m) 10s ago 2m 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 c819ca720db6 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.k smithi181 running (3m) 12s ago 3m 16.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 b8b89a131a6c 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mds.l smithi204 running (3m) 12s ago 3m 14.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 a3101db400f9 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.x smithi057 *:9283,8765 running (10m) 10s ago 10m 511M - 19.1.1-298-gb53b6d3e 54467abc55f8 c1d562973cb0 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.y smithi181 *:8443,8765 running (8m) 12s ago 8m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 51dc700a7d52 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mgr.z smithi204 *:8443,8765 running (8m) 12s ago 8m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 390ff4fc4855 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mon.a smithi057 running (10m) 10s ago 10m 50.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 6aafe956a410 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mon.b smithi181 running (9m) 12s ago 8m 44.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 2f6e4b2e383d 2024-09-12T03:05:25.034 INFO:teuthology.orchestra.run.smithi057.stdout:mon.c smithi204 running (9m) 12s ago 9m 43.9M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 312615cfb88f 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 smithi057 running (8m) 10s ago 8m 53.3M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 8840511dad4d 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 smithi181 running (7m) 12s ago 7m 54.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 efb7545e4921 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 smithi204 running (7m) 12s ago 7m 57.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 595de2cf67ea 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 smithi057 running (6m) 10s ago 6m 46.3M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 899efbaae52c 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 smithi181 running (6m) 12s ago 6m 43.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 faff571a5476 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 smithi204 running (6m) 12s ago 6m 45.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 5977f3161707 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 smithi057 running (5m) 10s ago 5m 41.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 a141222e7ee4 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 smithi181 running (5m) 12s ago 5m 45.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d8ec301c83b0 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 smithi204 running (4m) 12s ago 4m 51.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 bae891b3ea96 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 smithi057 running (4m) 10s ago 4m 41.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 b5f3c3207ccb 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 smithi181 running (3m) 12s ago 3m 52.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cb87ab63636d 2024-09-12T03:05:25.035 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 smithi204 running (3m) 12s ago 3m 43.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 df43a657e32d 2024-09-12T03:05:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 msgr2=0x7f90080a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 0x7f90080a5180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f902019b5c0 tx=0x7f9010005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 msgr2=0x7f902019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9014031440 tx=0x7f9014032ce0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 shutdown_connections 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f90080a2cc0 0x7f90080a5180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9020105920 0x7f90201a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9020104f60 0x7f902019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 --2- 172.21.15.57:0/3149311727 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9020103e00 0x7f902019a0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.034+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 >> 172.21.15.57:0/3149311727 conn(0x7f90200ff5b0 msgr2=0x7f9020108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.035+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 shutdown_connections 2024-09-12T03:05:25.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:25.035+0000 7f9028800640 1 -- 172.21.15.57:0/3149311727 wait complete. 2024-09-12T03:05:25.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:25 smithi057 ceph-mon[30361]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:25.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:25 smithi204 ceph-mon[35013]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:25 smithi181 ceph-mon[35320]: pgmap v378: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:25.800 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:05:26.177 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:26.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:26 smithi057 ceph-mon[30361]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:26.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:26 smithi204 ceph-mon[35013]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:26.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:26 smithi181 ceph-mon[35320]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 -- 172.21.15.57:0/29766815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 msgr2=0x7f6e00103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 --2- 172.21.15.57:0/29766815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e00103280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009900 tx=0x7f6de802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 -- 172.21.15.57:0/29766815 shutdown_connections 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 --2- 172.21.15.57:0/29766815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e0010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 --2- 172.21.15.57:0/29766815 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e00103280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:26.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 --2- 172.21.15.57:0/29766815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:26.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 -- 172.21.15.57:0/29766815 >> 172.21.15.57:0/29766815 conn(0x7f6e000fd390 msgr2=0x7f6e000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:26.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 -- 172.21.15.57:0/29766815 shutdown_connections 2024-09-12T03:05:26.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.788+0000 7f6e05600640 1 -- 172.21.15.57:0/29766815 wait complete. 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.789+0000 7f6e05600640 1 Processor -- start 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.789+0000 7f6e05600640 1 -- start start 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.789+0000 7f6e05600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6e05600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6e05600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e00074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:26.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6e05600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0010d880 con 0x7f6e00102e00 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6e05600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0010d700 con 0x7f6e001037c0 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6e05600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0010da00 con 0x7f6e00101c00 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46596/0 (socket says 172.21.15.57:46596) 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6df7e00640 1 -- 172.21.15.57:0/2326132012 learned_addr learned my addr 172.21.15.57:0/2326132012 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6dfea00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6dff400640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e00074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6df7e00640 1 -- 172.21.15.57:0/2326132012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 msgr2=0x7f6e00075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.790+0000 7f6df7e00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7e00640 1 -- 172.21.15.57:0/2326132012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 msgr2=0x7f6e00074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7e00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e00074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7e00640 1 -- 172.21.15.57:0/2326132012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e000753e0 con 0x7f6e00102e00 2024-09-12T03:05:26.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6dfea00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6dff400640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e00074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7e00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009230 tx=0x7f6de8031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de803f030 con 0x7f6e00102e00 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e001a2070 con 0x7f6e00102e00 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de8003ad0 con 0x7f6e00102e00 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8049060 con 0x7f6e00102e00 2024-09-12T03:05:26.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.791+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e001a2570 con 0x7f6e00102e00 2024-09-12T03:05:26.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.793+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e00102000 con 0x7f6e00102e00 2024-09-12T03:05:26.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.794+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6de8042030 con 0x7f6e00102e00 2024-09-12T03:05:26.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.796+0000 7f6df7400640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 0x7f6de40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:26.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.796+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f6de80efea0 con 0x7f6e00102e00 2024-09-12T03:05:26.796 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.796+0000 7f6dfea00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 0x7f6de40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:26.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.797+0000 7f6dfea00640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 0x7f6de40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e00102c60 tx=0x7f6df0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:26.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.800+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de80bc220 con 0x7f6e00102e00 2024-09-12T03:05:26.992 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.992+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6e000626f0 con 0x7f6de40a2c10 2024-09-12T03:05:26.998 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:26.998+0000 7f6df7400640 1 -- 172.21.15.57:0/2326132012 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6e000626f0 con 0x7f6de40a2c10 2024-09-12T03:05:26.998 INFO:teuthology.orchestra.run.smithi057.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:05:26.998 INFO:teuthology.orchestra.run.smithi057.stdout:mds.all 12/12 14s ago 3m smithi057=a;smithi057=d;smithi057=g;smithi057=j;smithi181=b;smithi181=e;smithi181=h;smithi181=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-12T03:05:26.998 INFO:teuthology.orchestra.run.smithi057.stdout:mgr 3/3 14s ago 8m smithi057=x;smithi181=y;smithi204=z;count:3 2024-09-12T03:05:26.999 INFO:teuthology.orchestra.run.smithi057.stdout:mon 3/3 14s ago 9m smithi057:172.21.15.57=a;smithi181:172.21.15.181=b;smithi204:172.21.15.204=c;count:3 2024-09-12T03:05:26.999 INFO:teuthology.orchestra.run.smithi057.stdout:osd 12 14s ago - 2024-09-12T03:05:27.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 msgr2=0x7f6de40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 0x7f6de40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6e00102c60 tx=0x7f6df0005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 msgr2=0x7f6e000745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009230 tx=0x7f6de8031b60 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 shutdown_connections 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f6de40a2c10 0x7f6de40a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e001037c0 0x7f6e00074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6e00102e00 0x7f6e000745e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 --2- 172.21.15.57:0/2326132012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101c00 0x7f6e00075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 >> 172.21.15.57:0/2326132012 conn(0x7f6e000fd390 msgr2=0x7f6e00106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 shutdown_connections 2024-09-12T03:05:27.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:27.000+0000 7f6e05600640 1 -- 172.21.15.57:0/2326132012 wait complete. 2024-09-12T03:05:27.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:27 smithi057 ceph-mon[30361]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:27.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:27 smithi204 ceph-mon[35013]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:27 smithi181 ceph-mon[35320]: pgmap v379: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:27.876 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:05:28.255 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:28.575 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:28 smithi057 ceph-mon[30361]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:28.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:28 smithi204 ceph-mon[35013]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:28.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:28 smithi181 ceph-mon[35320]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 -- 172.21.15.57:0/2715973073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 msgr2=0x7f94ac101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 --2- 172.21.15.57:0/2715973073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac101e40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009900 tx=0x7f94a002a410 comp rx=0 tx=0).stop 2024-09-12T03:05:28.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 -- 172.21.15.57:0/2715973073 shutdown_connections 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 --2- 172.21.15.57:0/2715973073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 0x7f94ac109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 --2- 172.21.15.57:0/2715973073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 0x7f94ac1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 --2- 172.21.15.57:0/2715973073 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac101e40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.830+0000 7f94b2600640 1 -- 172.21.15.57:0/2715973073 >> 172.21.15.57:0/2715973073 conn(0x7f94ac0fd1f0 msgr2=0x7f94ac0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.831+0000 7f94b2600640 1 -- 172.21.15.57:0/2715973073 shutdown_connections 2024-09-12T03:05:28.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.831+0000 7f94b2600640 1 -- 172.21.15.57:0/2715973073 wait complete. 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 Processor -- start 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 -- start start 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 0x7f94ac19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 0x7f94ac1a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac10d6d0 con 0x7f94ac101a40 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac10d550 con 0x7f94ac102c40 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac10d850 con 0x7f94ac103600 2024-09-12T03:05:28.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 0x7f94ac19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44806/0 (socket says 172.21.15.57:44806) 2024-09-12T03:05:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b1200640 1 -- 172.21.15.57:0/2421524459 learned_addr learned my addr 172.21.15.57:0/2421524459 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:28.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.832+0000 7f94b1c00640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 0x7f94ac1a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 -- 172.21.15.57:0/2421524459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 msgr2=0x7f94ac1a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 0x7f94ac1a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 -- 172.21.15.57:0/2421524459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 msgr2=0x7f94ac19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 0x7f94ac19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 -- 172.21.15.57:0/2421524459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac1a1b10 con 0x7f94ac101a40 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.833+0000 7f94b1200640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031610 tx=0x7f94a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.834+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a003f030 con 0x7f94ac101a40 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.834+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a0003ad0 con 0x7f94ac101a40 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.834+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac1a1da0 con 0x7f94ac101a40 2024-09-12T03:05:28.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.834+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0035070 con 0x7f94ac101a40 2024-09-12T03:05:28.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.834+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a21c0 con 0x7f94ac101a40 2024-09-12T03:05:28.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.835+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac1a23f0 con 0x7f94ac101a40 2024-09-12T03:05:28.837 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.837+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f94a0042030 con 0x7f94ac101a40 2024-09-12T03:05:28.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.838+0000 7f94a9600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 0x7f94940a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:28.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.838+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f94a00ef810 con 0x7f94ac101a40 2024-09-12T03:05:28.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.838+0000 7f94abe00640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 0x7f94940a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:28.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.839+0000 7f94abe00640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 0x7f94940a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94980095f0 tx=0x7f9498005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:28.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:28.842+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94a00bbb90 con 0x7f94ac101a40 2024-09-12T03:05:29.034 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.034+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f94ac109b80 con 0x7f94940a2c40 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.035+0000 7f94a9600640 1 -- 172.21.15.57:0/2421524459 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f94ac109b80 con 0x7f94940a2c40 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 172.21.15.57 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stdout:smithi181 172.21.15.181 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stdout:smithi204 172.21.15.204 2024-09-12T03:05:29.036 INFO:teuthology.orchestra.run.smithi057.stdout:3 hosts in cluster 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 msgr2=0x7f94940a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 0x7f94940a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94980095f0 tx=0x7f9498005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 msgr2=0x7f94ac199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f94a0031610 tx=0x7f94a0004270 comp rx=0 tx=0).stop 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 shutdown_connections 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f94940a2c40 0x7f94940a5100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:29.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94ac103600 0x7f94ac1a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:29.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac102c40 0x7f94ac19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:29.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 --2- 172.21.15.57:0/2421524459 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f94ac101a40 0x7f94ac199ee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:29.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 >> 172.21.15.57:0/2421524459 conn(0x7f94ac0fd1f0 msgr2=0x7f94ac108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:29.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 shutdown_connections 2024-09-12T03:05:29.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:29.038+0000 7f94b2600640 1 -- 172.21.15.57:0/2421524459 wait complete. 2024-09-12T03:05:29.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:29 smithi057 ceph-mon[30361]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:29.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:29 smithi204 ceph-mon[35013]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:29 smithi181 ceph-mon[35320]: pgmap v380: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:29.860 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:05:30.237 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:30.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:30 smithi057 ceph-mon[30361]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:30.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:30 smithi204 ceph-mon[35013]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:30.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:30 smithi181 ceph-mon[35320]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 -- 172.21.15.57:0/2325004913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa8076220 msgr2=0x7f5fa8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 --2- 172.21.15.57:0/2325004913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa8076220 0x7f5fa8074680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009900 tx=0x7f5f9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 -- 172.21.15.57:0/2325004913 shutdown_connections 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 --2- 172.21.15.57:0/2325004913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa810e1e0 0x7f5fa81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 --2- 172.21.15.57:0/2325004913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8074c50 0x7f5fa80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 --2- 172.21.15.57:0/2325004913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa8076220 0x7f5fa8074680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.792+0000 7f5fafc00640 1 -- 172.21.15.57:0/2325004913 >> 172.21.15.57:0/2325004913 conn(0x7f5fa80fd390 msgr2=0x7f5fa80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.793+0000 7f5fafc00640 1 -- 172.21.15.57:0/2325004913 shutdown_connections 2024-09-12T03:05:30.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.793+0000 7f5fafc00640 1 -- 172.21.15.57:0/2325004913 wait complete. 2024-09-12T03:05:30.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.794+0000 7f5fafc00640 1 Processor -- start 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.794+0000 7f5fafc00640 1 -- start start 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.794+0000 7f5fafc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa8074c50 0x7f5fa819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.794+0000 7f5fafc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fafc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa810e1e0 0x7f5fa81a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fafc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8113ef0 con 0x7f5fa810e1e0 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fafc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8113d70 con 0x7f5fa8074c50 2024-09-12T03:05:30.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fafc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8114070 con 0x7f5fa8076220 2024-09-12T03:05:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fadc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa810e1e0 0x7f5fa81a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48830/0 (socket says 172.21.15.57:48830) 2024-09-12T03:05:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 -- 172.21.15.57:0/1892340157 learned_addr learned my addr 172.21.15.57:0/1892340157 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:30.799 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fad200640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa8074c50 0x7f5fa819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 -- 172.21.15.57:0/1892340157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa8074c50 msgr2=0x7f5fa819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa8074c50 0x7f5fa819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 -- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa810e1e0 msgr2=0x7f5fa81a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa810e1e0 0x7f5fa81a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.795+0000 7f5fa7e00640 1 -- 172.21.15.57:0/1892340157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa81a61e0 con 0x7f5fa8076220 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fa7e00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e9f0 tx=0x7f5f9400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400cd80 con 0x7f5fa8076220 2024-09-12T03:05:30.800 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa81a64d0 con 0x7f5fa8076220 2024-09-12T03:05:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f94005500 con 0x7f5fa8076220 2024-09-12T03:05:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94010690 con 0x7f5fa8076220 2024-09-12T03:05:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.796+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa81a6a10 con 0x7f5fa8076220 2024-09-12T03:05:30.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.797+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa8103b80 con 0x7f5fa8076220 2024-09-12T03:05:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.799+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5f940046c0 con 0x7f5fa8076220 2024-09-12T03:05:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.800+0000 7f5fa5600640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 0x7f5f900a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:30.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.801+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5f940cf920 con 0x7f5fa8076220 2024-09-12T03:05:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.801+0000 7f5fad200640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 0x7f5f900a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.801+0000 7f5fad200640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 0x7f5f900a5180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c02aa00 tx=0x7f5f9c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:30.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:30.802+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f9409c2f0 con 0x7f5fa8076220 2024-09-12T03:05:31.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.001+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5fa8062630 con 0x7f5f900a2cc0 2024-09-12T03:05:31.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.003+0000 7f5fa5600640 1 -- 172.21.15.57:0/1892340157 <== mgr.14150 v2:172.21.15.57:6800/480086157 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5fa8062630 con 0x7f5f900a2cc0 2024-09-12T03:05:31.004 INFO:teuthology.orchestra.run.smithi057.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:05:31.004 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE7361049H4P0KGN 3726G No 4m ago Has a FileSystem, LVM detected 2024-09-12T03:05:31.004 INFO:teuthology.orchestra.run.smithi057.stdout:smithi181 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000E400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:05:31.004 INFO:teuthology.orchestra.run.smithi057.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:05:31.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.005+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 msgr2=0x7f5f900a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:31.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.005+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 0x7f5f900a5180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c02aa00 tx=0x7f5f9c0315b0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.005+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 msgr2=0x7f5fa819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:31.006 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.005+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e9f0 tx=0x7f5f9400edf0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 shutdown_connections 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5f900a2cc0 0x7f5f900a5180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5fa810e1e0 0x7f5fa81a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fa8076220 0x7f5fa819eaf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 --2- 172.21.15.57:0/1892340157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fa8074c50 0x7f5fa819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 >> 172.21.15.57:0/1892340157 conn(0x7f5fa80fd390 msgr2=0x7f5fa80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 shutdown_connections 2024-09-12T03:05:31.007 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:31.006+0000 7f5fafc00640 1 -- 172.21.15.57:0/1892340157 wait complete. 2024-09-12T03:05:31.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:31 smithi057 ceph-mon[30361]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:31.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:31 smithi204 ceph-mon[35013]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:31 smithi181 ceph-mon[35320]: pgmap v381: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:31.936 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:05:31.945 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:05:31.945 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:05:32.323 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:32.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:32 smithi057 ceph-mon[30361]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:32.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:32 smithi204 ceph-mon[35013]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:32.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:32 smithi181 ceph-mon[35320]: from='client.25072 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 -- 172.21.15.57:0/3008308390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 msgr2=0x7f5934104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 --2- 172.21.15.57:0/3008308390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934104050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5924009900 tx=0x7f592402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 -- 172.21.15.57:0/3008308390 shutdown_connections 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 --2- 172.21.15.57:0/3008308390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934105810 0x7f593410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 --2- 172.21.15.57:0/3008308390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934104e50 0x7f59341052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 --2- 172.21.15.57:0/3008308390 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934104050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.944+0000 7f5933e00640 1 -- 172.21.15.57:0/3008308390 >> 172.21.15.57:0/3008308390 conn(0x7f59340ff3e0 msgr2=0x7f5934101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.945+0000 7f5933e00640 1 -- 172.21.15.57:0/3008308390 shutdown_connections 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.945+0000 7f5933e00640 1 -- 172.21.15.57:0/3008308390 wait complete. 2024-09-12T03:05:32.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.945+0000 7f5933e00640 1 Processor -- start 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 -- start start 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934104e50 0x7f593419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f593410f8d0 con 0x7f5934103c50 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f593410f750 con 0x7f5934104e50 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f593410fa50 con 0x7f5934105810 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5932a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48842/0 (socket says 172.21.15.57:48842) 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5933400640 1 -- 172.21.15.57:0/1643703809 learned_addr learned my addr 172.21.15.57:0/1643703809 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:32.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5932000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934104e50 0x7f593419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.946+0000 7f5932a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44848/0 (socket says 172.21.15.57:44848) 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 -- 172.21.15.57:0/1643703809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934104e50 msgr2=0x7f593419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934104e50 0x7f593419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 -- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 msgr2=0x7f5934199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 -- 172.21.15.57:0/1643703809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59341a1af0 con 0x7f5934105810 2024-09-12T03:05:32.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.947+0000 7f5933400640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f592c00cce0 tx=0x7f592c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.948+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00f450 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.948+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c00fad0 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.948+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59341a1de0 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.948+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c018c00 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.948+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341a2240 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.949+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593410b000 con 0x7f5934105810 2024-09-12T03:05:32.951 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.951+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f592c00fc80 con 0x7f5934105810 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.952+0000 7f5938a00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 0x7f59180a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:32.953 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.953+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f592c0cfbc0 con 0x7f5934105810 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.953+0000 7f5932a00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 0x7f59180a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:32.954 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.953+0000 7f5932a00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 0x7f59180a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5924005ab0 tx=0x7f5924000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:32.955 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:32.955+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f592c014030 con 0x7f5934105810 2024-09-12T03:05:33.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.174+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f593402da10 con 0x7f5934105810 2024-09-12T03:05:33.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.175+0000 7f5938a00640 1 -- 172.21.15.57:0/1643703809 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3096 (secure 0 0 0) 0x7f592c09c590 con 0x7f5934105810 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:e25 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:btime 2024-09-12T03:03:59:812711+0000 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:legacy client fscid: 1 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:33.176 INFO:teuthology.orchestra.run.smithi057.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:fs_name cephfs 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 25 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-09-12T03:03:08.679612+0000 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-09-12T03:03:58.893616+0000 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:tableserver 0 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:root 0 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:session_timeout 300 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:session_autoclose 300 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:max_file_size 1099511627776 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:max_xattr_size 65536 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:required_client_features {} 2024-09-12T03:05:33.177 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure 0 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:last_failure_osd_epoch 0 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:max_mds 3 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:in 0,1,2 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:up {0=24554,1=24548,2=24529} 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:failed 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:damaged 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:stopped 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:data_pools [3,4] 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:metadata_pool 2 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:inline_data disabled 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:balancer 2024-09-12T03:05:33.178 INFO:teuthology.orchestra.run.smithi057.stdout:bal_rank_mask -1 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:standby_count_wanted 0 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:qdb_cluster leader: 24554 members: 24529,24548,24554 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.b{0:24554} state up:active seq 5 addr [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.h{0:24515} state up:standby-replay seq 1 addr [v2:172.21.15.181:6836/2458916124,v1:172.21.15.181:6837/2458916124] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.a{1:24548} state up:active seq 17 addr [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.c{1:24511} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/368351857,v1:172.21.15.204:6837/368351857] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.i{2:24529} state up:active seq 19 addr [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.f{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/1804109241,v1:172.21.15.204:6835/1804109241] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:Standby daemons: 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:33.179 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.g{-1:14490} state up:standby seq 1 addr [v2:172.21.15.57:6838/1375693587,v1:172.21.15.57:6839/1375693587] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.k{-1:24455} state up:standby seq 1 addr [v2:172.21.15.181:6832/1889756926,v1:172.21.15.181:6833/1889756926] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.204:6832/3741354654,v1:172.21.15.204:6833/3741354654] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.j{-1:24467} state up:standby seq 1 addr [v2:172.21.15.57:6834/4044082469,v1:172.21.15.57:6835/4044082469] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.d{-1:24479} state up:standby seq 1 addr [v2:172.21.15.57:6836/3595237430,v1:172.21.15.57:6837/3595237430] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stdout:[mds.e{-1:24491} state up:standby seq 1 addr [v2:172.21.15.181:6834/1691601828,v1:172.21.15.181:6835/1691601828] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:05:33.180 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 msgr2=0x7f59180a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 0x7f59180a5100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5924005ab0 tx=0x7f5924000bc0 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 msgr2=0x7f59341a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f592c00cce0 tx=0x7f592c00a620 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 shutdown_connections 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f59180a2c40 0x7f59180a5100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5934105810 0x7f59341a1480 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5934104e50 0x7f593419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 --2- 172.21.15.57:0/1643703809 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5934103c50 0x7f5934199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.177+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 >> 172.21.15.57:0/1643703809 conn(0x7f59340ff3e0 msgr2=0x7f59341081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.178+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 shutdown_connections 2024-09-12T03:05:33.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:33.178+0000 7f5933e00640 1 -- 172.21.15.57:0/1643703809 wait complete. 2024-09-12T03:05:33.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:33 smithi057 ceph-mon[30361]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:33.602 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:33 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1643703809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:05:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:33 smithi204 ceph-mon[35013]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:33 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1643703809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:05:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:33 smithi181 ceph-mon[35320]: pgmap v382: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:05:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:33 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1643703809' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:05:33.983 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:05:34.363 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:35.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 -- 172.21.15.57:0/3221971834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 msgr2=0x7f5ae4110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/3221971834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae4110410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009ca0 tx=0x7f5adc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 -- 172.21.15.57:0/3221971834 shutdown_connections 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/3221971834 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae4110410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/3221971834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 0x7f5ae40750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/3221971834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae4074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 -- 172.21.15.57:0/3221971834 >> 172.21.15.57:0/3221971834 conn(0x7f5ae40fd1f0 msgr2=0x7f5ae40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 -- 172.21.15.57:0/3221971834 shutdown_connections 2024-09-12T03:05:35.010 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.009+0000 7f5aeaa00640 1 -- 172.21.15.57:0/3221971834 wait complete. 2024-09-12T03:05:35.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 Processor -- start 2024-09-12T03:05:35.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 -- start start 2024-09-12T03:05:35.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 0x7f5ae419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:35.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:35.011 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae41a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4113d30 con 0x7f5ae410e020 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4113bb0 con 0x7f5ae4074c50 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.010+0000 7f5aeaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4113eb0 con 0x7f5ae4076220 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5aea000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae41a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48868/0 (socket says 172.21.15.57:48868) 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 -- 172.21.15.57:0/2479092926 learned_addr learned my addr 172.21.15.57:0/2479092926 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:35.012 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae9600640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 0x7f5ae419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 -- 172.21.15.57:0/2479092926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 msgr2=0x7f5ae419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 0x7f5ae419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 -- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 msgr2=0x7f5ae41a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae41a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 -- 172.21.15.57:0/2479092926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41a5ec0 con 0x7f5ae4076220 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5aea000640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae41a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.011+0000 7f5ae8c00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000e9f0 tx=0x7f5ad000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:35.013 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.012+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000cd80 con 0x7f5ae4076220 2024-09-12T03:05:35.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.012+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae41a61b0 con 0x7f5ae4076220 2024-09-12T03:05:35.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.012+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ad0010910 con 0x7f5ae4076220 2024-09-12T03:05:35.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.012+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00188f0 con 0x7f5ae4076220 2024-09-12T03:05:35.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.012+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41a6740 con 0x7f5ae4076220 2024-09-12T03:05:35.014 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.013+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae41040d0 con 0x7f5ae4076220 2024-09-12T03:05:35.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.015+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ad0010ac0 con 0x7f5ae4076220 2024-09-12T03:05:35.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.016+0000 7f5ae2000640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 0x7f5acc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:35.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.017+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f5ad00cfd40 con 0x7f5ae4076220 2024-09-12T03:05:35.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.017+0000 7f5ae9600640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 0x7f5acc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:35.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.017+0000 7f5ae9600640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 0x7f5acc0a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4076600 tx=0x7f5ad8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:35.019 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.018+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ad009c0c0 con 0x7f5ae4076220 2024-09-12T03:05:35.206 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.206+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5ae402d9c0 con 0x7f5ae4076220 2024-09-12T03:05:35.207 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.207+0000 7f5ae2000640 1 -- 172.21.15.57:0/2479092926 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) ==== 56+0+5694 (secure 0 0 0) 0x7f5ad00a03c0 con 0x7f5ae4076220 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:epoch 96 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:created 2024-09-12T02:54:45.449832+0000 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:modified 2024-09-12T03:04:42.450539+0000 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:crush_version 27 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:full_ratio 0.95 2024-09-12T03:05:35.208 INFO:teuthology.orchestra.run.smithi057.stdout:backfillfull_ratio 0.9 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:nearfull_ratio 0.85 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:require_min_compat_client luminous 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:min_compat_client luminous 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:require_osd_release squid 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:stretch_mode_enabled false 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-12T03:05:35.209 INFO:teuthology.orchestra.run.smithi057.stdout:max_osd 12 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6802/3978791581,v1:172.21.15.57:6803/3978791581] [v2:172.21.15.57:6804/3978791581,v1:172.21.15.57:6805/3978791581] exists,up 32912ece-df23-430a-95fe-259e65d98666 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.1 up in weight 1 up_from 13 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6800/1533372313,v1:172.21.15.181:6801/1533372313] [v2:172.21.15.181:6802/1533372313,v1:172.21.15.181:6803/1533372313] exists,up 77ea4ec1-f61c-4839-b2c4-453c0f8e2415 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/82472683,v1:172.21.15.204:6801/82472683] [v2:172.21.15.204:6802/82472683,v1:172.21.15.204:6803/82472683] exists,up 6af3e4b4-81a9-4247-b17e-ab3ca353d09b 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.3 up in weight 1 up_from 24 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6810/2741769998,v1:172.21.15.57:6811/2741769998] [v2:172.21.15.57:6812/2741769998,v1:172.21.15.57:6813/2741769998] exists,up 4746f9bf-15b6-4dfb-8e93-8140a669499e 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.4 up in weight 1 up_from 32 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6808/3730474158,v1:172.21.15.181:6809/3730474158] [v2:172.21.15.181:6810/3730474158,v1:172.21.15.181:6811/3730474158] exists,up 9ed45fbb-6c70-460b-8f77-77f8810b2e0e 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.5 up in weight 1 up_from 37 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/4265194812,v1:172.21.15.204:6809/4265194812] [v2:172.21.15.204:6810/4265194812,v1:172.21.15.204:6811/4265194812] exists,up 08fe7507-dac1-4941-b250-aa6352886ca7 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.6 up in weight 1 up_from 45 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6818/1487516632,v1:172.21.15.57:6819/1487516632] [v2:172.21.15.57:6820/1487516632,v1:172.21.15.57:6821/1487516632] exists,up 791b2b8c-0c86-49ca-a977-e6cb195ad8ae 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.7 up in weight 1 up_from 50 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6816/159674330,v1:172.21.15.181:6817/159674330] [v2:172.21.15.181:6818/159674330,v1:172.21.15.181:6819/159674330] exists,up 45285c14-5770-46e0-8eb8-8f8e46d74882 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.8 up in weight 1 up_from 57 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/2786968053,v1:172.21.15.204:6817/2786968053] [v2:172.21.15.204:6818/2786968053,v1:172.21.15.204:6819/2786968053] exists,up 2012481a-b076-4538-a4f4-09c1e17f89f4 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.9 up in weight 1 up_from 65 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.57:6826/3989705460,v1:172.21.15.57:6827/3989705460] [v2:172.21.15.57:6828/3989705460,v1:172.21.15.57:6829/3989705460] exists,up bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.10 up in weight 1 up_from 70 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.181:6824/25222354,v1:172.21.15.181:6825/25222354] [v2:172.21.15.181:6826/25222354,v1:172.21.15.181:6827/25222354] exists,up 46a48ee5-1571-4aa2-91ed-2f972739a137 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:osd.11 up in weight 1 up_from 77 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/1683140882,v1:172.21.15.204:6825/1683140882] [v2:172.21.15.204:6826/1683140882,v1:172.21.15.204:6827/1683140882] exists,up 14d16ce7-b559-4958-9d46-03684f5c25ca 2024-09-12T03:05:35.210 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.0 [8,4] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.8 [2,11] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.b [8,11] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 2.16 [2,4] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.e [4,6] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.1a [5,7] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.25 [5,9] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.29 [3,9] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.2c [4,9] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 3.31 [4,8] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.2 [9,1] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.e [5,8] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.1b [9,8] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:pg_upmap_items 4.1c [9,8] 2024-09-12T03:05:35.211 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1578458239 expires 2024-09-13T02:55:39.619488+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/522425633 expires 2024-09-13T02:55:39.619488+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6801/1326814975 expires 2024-09-13T02:55:10.466284+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/1326814975 expires 2024-09-13T02:55:10.466284+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1345986708 expires 2024-09-13T02:55:10.466284+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1176788560 expires 2024-09-13T02:55:39.619488+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:6800/522425633 expires 2024-09-13T02:55:39.619488+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/4021182045 expires 2024-09-13T02:55:10.466284+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/752811668 expires 2024-09-13T02:55:39.619488+0000 2024-09-12T03:05:35.212 INFO:teuthology.orchestra.run.smithi057.stdout:blocklist 172.21.15.57:0/1256058423 expires 2024-09-13T02:55:10.466284+0000 2024-09-12T03:05:35.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 msgr2=0x7f5acc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:35.213 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 0x7f5acc0a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4076600 tx=0x7f5ad8005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 msgr2=0x7f5ae419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000e9f0 tx=0x7f5ad000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 shutdown_connections 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f5acc0a2c40 0x7f5acc0a5100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ae410e020 0x7f5ae41a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ae4076220 0x7f5ae419e7d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.214 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 --2- 172.21.15.57:0/2479092926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ae4074c50 0x7f5ae419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:35.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 >> 172.21.15.57:0/2479092926 conn(0x7f5ae40fd1f0 msgr2=0x7f5ae41012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:35.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.209+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 shutdown_connections 2024-09-12T03:05:35.215 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:35.210+0000 7f5aeaa00640 1 -- 172.21.15.57:0/2479092926 wait complete. 2024-09-12T03:05:35.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:35 smithi057 ceph-mon[30361]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:35.608 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:35 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2479092926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:05:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:35 smithi204 ceph-mon[35013]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:35 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2479092926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:05:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:35 smithi181 ceph-mon[35320]: pgmap v383: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T03:05:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:35 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2479092926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:05:36.040 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:05:36.053 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:36.432 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1777160573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074620 msgr2=0x7f2bf0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1777160573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074620 0x7f2bf0074a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009900 tx=0x7f2be002a480 comp rx=0 tx=0).stop 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1777160573 shutdown_connections 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1777160573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf010e3c0 0x7f2bf0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1777160573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf0074f40 0x7f2bf010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1777160573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074620 0x7f2bf0074a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1777160573 >> 172.21.15.57:0/1777160573 conn(0x7f2bf00fd8c0 msgr2=0x7f2bf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1777160573 shutdown_connections 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.991+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1777160573 wait complete. 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 Processor -- start 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 -- start start 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf0074620 0x7f2bf019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.016 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf010e3c0 0x7f2bf019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0114170 con 0x7f2bf0074f40 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0113ff0 con 0x7f2bf010e3c0 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.992+0000 7f2bf4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01142f0 con 0x7f2bf0074620 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44884/0 (socket says 172.21.15.57:44884) 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bede00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf0074620 0x7f2bf019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 -- 172.21.15.57:0/1809822391 learned_addr learned my addr 172.21.15.57:0/1809822391 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bee800640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf010e3c0 0x7f2bf019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 -- 172.21.15.57:0/1809822391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf0074620 msgr2=0x7f2bf019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf0074620 0x7f2bf019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 -- 172.21.15.57:0/1809822391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf010e3c0 msgr2=0x7f2bf019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf010e3c0 0x7f2bf019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.017 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 -- 172.21.15.57:0/1809822391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf019f660 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.993+0000 7f2bed400640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9f0 tx=0x7f2bdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.994+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00cd80 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.994+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bdc010910 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.994+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf019f950 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.994+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc018990 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.994+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a7230 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.995+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0074a00 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.997+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2bdc010ac0 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.998+0000 7f2be6a00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 0x7f2bd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.998+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2bdc0cfe80 con 0x7f2bf0074f40 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.998+0000 7f2bede00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 0x7f2bd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:36.999+0000 7f2bede00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 0x7f2bd40a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009230 tx=0x7f2be00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:37.018 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.001+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bdc09c200 con 0x7f2bf0074f40 2024-09-12T03:05:37.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.223+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bf002d7c0 con 0x7f2bf0074f40 2024-09-12T03:05:37.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.224+0000 7f2be6a00640 1 -- 172.21.15.57:0/1809822391 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7f2bdc0a0500 con 0x7f2bf0074f40 2024-09-12T03:05:37.225 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:37.225 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:05:37.226 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 msgr2=0x7f2bd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 0x7f2bd40a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009230 tx=0x7f2be00058d0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 msgr2=0x7f2bf019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9f0 tx=0x7f2bdc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 shutdown_connections 2024-09-12T03:05:37.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2bd40a2c10 0x7f2bd40a50d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bf010e3c0 0x7f2bf019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2bf0074f40 0x7f2bf019abf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.227+0000 7f2bf4e00640 1 --2- 172.21.15.57:0/1809822391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf0074620 0x7f2bf019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.228+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 >> 172.21.15.57:0/1809822391 conn(0x7f2bf00fd8c0 msgr2=0x7f2bf00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.228+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 shutdown_connections 2024-09-12T03:05:37.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:37.228+0000 7f2bf4e00640 1 -- 172.21.15.57:0/1809822391 wait complete. 2024-09-12T03:05:37.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:37 smithi057 ceph-mon[30361]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:37 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1809822391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:37 smithi204 ceph-mon[35013]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:37 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1809822391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:37 smithi181 ceph-mon[35320]: pgmap v384: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T03:05:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:37 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1809822391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:38.106 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:05:38.485 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 -- 172.21.15.57:0/301697526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8109140 msgr2=0x7f23c8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 --2- 172.21.15.57:0/301697526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8109140 0x7f23c8110cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23c0009be0 tx=0x7f23c002a6c0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 -- 172.21.15.57:0/301697526 shutdown_connections 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 --2- 172.21.15.57:0/301697526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8109140 0x7f23c8110cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 --2- 172.21.15.57:0/301697526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 0x7f23c8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 --2- 172.21.15.57:0/301697526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8104010 0x7f23c81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 -- 172.21.15.57:0/301697526 >> 172.21.15.57:0/301697526 conn(0x7f23c80ffa50 msgr2=0x7f23c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:39.102 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 -- 172.21.15.57:0/301697526 shutdown_connections 2024-09-12T03:05:39.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.101+0000 7f23cd400640 1 -- 172.21.15.57:0/301697526 wait complete. 2024-09-12T03:05:39.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.102+0000 7f23cd400640 1 Processor -- start 2024-09-12T03:05:39.103 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.102+0000 7f23cd400640 1 -- start start 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 0x7f23c819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 0x7f23c819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8114100 con 0x7f23c8109140 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8113f80 con 0x7f23c81049c0 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23cd400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23c8114280 con 0x7f23c8104010 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23c7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23c7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48550/0 (socket says 172.21.15.57:48550) 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23c7200640 1 -- 172.21.15.57:0/1632516871 learned_addr learned my addr 172.21.15.57:0/1632516871 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23c6800640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 0x7f23c819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:39.104 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.103+0000 7f23bde00640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 0x7f23c819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:39.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 -- 172.21.15.57:0/1632516871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 msgr2=0x7f23c819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:39.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 0x7f23c819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 -- 172.21.15.57:0/1632516871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 msgr2=0x7f23c819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:39.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 0x7f23c819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.105 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 -- 172.21.15.57:0/1632516871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c819f570 con 0x7f23c8109140 2024-09-12T03:05:39.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c6800640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 0x7f23c819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:39.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23c7200640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23c0034040 tx=0x7f23c00305b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:39.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0041020 con 0x7f23c8109140 2024-09-12T03:05:39.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c0037040 con 0x7f23c8109140 2024-09-12T03:05:39.106 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c819f800 con 0x7f23c8109140 2024-09-12T03:05:39.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.104+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0036070 con 0x7f23c8109140 2024-09-12T03:05:39.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.105+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c81a7140 con 0x7f23c8109140 2024-09-12T03:05:39.107 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.105+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c81043f0 con 0x7f23c8109140 2024-09-12T03:05:39.108 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.108+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23c0044020 con 0x7f23c8109140 2024-09-12T03:05:39.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.109+0000 7f23bfe00640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 0x7f23ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:39.109 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.109+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f23c00f03d0 con 0x7f23c8109140 2024-09-12T03:05:39.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.109+0000 7f23c6800640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 0x7f23ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:39.110 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.110+0000 7f23c6800640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 0x7f23ac0a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23b80096c0 tx=0x7f23b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:39.111 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.111+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23c004b020 con 0x7f23c8109140 2024-09-12T03:05:39.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.331+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23c810c3e0 con 0x7f23c8109140 2024-09-12T03:05:39.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.332+0000 7f23bfe00640 1 -- 172.21.15.57:0/1632516871 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11923 (secure 0 0 0) 0x7f23c00bcda0 con 0x7f23c8109140 2024-09-12T03:05:39.333 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:39.333 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":25,"btime":"2024-09-12T03:03:59:812711+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:03:58.893616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:05:39.333 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 25 2024-09-12T03:05:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.334+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 msgr2=0x7f23ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.334+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 0x7f23ac0a5150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f23b80096c0 tx=0x7f23b8009260 comp rx=0 tx=0).stop 2024-09-12T03:05:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.334+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 msgr2=0x7f23c819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:39.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.334+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23c0034040 tx=0x7f23c00305b0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 shutdown_connections 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f23ac0a2c90 0x7f23ac0a5150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f23c8109140 0x7f23c819ee90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23c81049c0 0x7f23c819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 --2- 172.21.15.57:0/1632516871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c8104010 0x7f23c819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 >> 172.21.15.57:0/1632516871 conn(0x7f23c80ffa50 msgr2=0x7f23c8100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 shutdown_connections 2024-09-12T03:05:39.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:39.335+0000 7f23cd400640 1 -- 172.21.15.57:0/1632516871 wait complete. 2024-09-12T03:05:39.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:39 smithi057 ceph-mon[30361]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:39 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1632516871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:39.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:39 smithi204 ceph-mon[35013]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:39.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:39 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1632516871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:39 smithi181 ceph-mon[35320]: pgmap v385: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T03:05:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:39 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1632516871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:05:40.181 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:05:40.565 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 -- 172.21.15.57:0/1477695241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a201047e0 msgr2=0x7f3a20108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/1477695241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a201047e0 0x7f3a20108830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009900 tx=0x7f3a0802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 -- 172.21.15.57:0/1477695241 shutdown_connections 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/1477695241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 0x7f3a20110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/1477695241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a201047e0 0x7f3a20108830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/1477695241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a20103e30 0x7f3a20104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 -- 172.21.15.57:0/1477695241 >> 172.21.15.57:0/1477695241 conn(0x7f3a200ff8d0 msgr2=0x7f3a20101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.172 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 -- 172.21.15.57:0/1477695241 shutdown_connections 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.168+0000 7f3a1fe00640 1 -- 172.21.15.57:0/1477695241 wait complete. 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.169+0000 7f3a1fe00640 1 Processor -- start 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.169+0000 7f3a1fe00640 1 -- start start 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a201047e0 0x7f3a2019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 0x7f3a2019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20113fc0 con 0x7f3a20103e30 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20113e40 con 0x7f3a20108f60 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20114140 con 0x7f3a201047e0 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48576/0 (socket says 172.21.15.57:48576) 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 -- 172.21.15.57:0/366493067 learned_addr learned my addr 172.21.15.57:0/366493067 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:41.173 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1f400640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 0x7f3a2019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a16000640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a201047e0 0x7f3a2019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 -- 172.21.15.57:0/366493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a201047e0 msgr2=0x7f3a2019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a201047e0 0x7f3a2019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 -- 172.21.15.57:0/366493067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 msgr2=0x7f3a2019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 0x7f3a2019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.170+0000 7f3a1ea00640 1 -- 172.21.15.57:0/366493067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019f430 con 0x7f3a20103e30 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a1ea00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ebc0 tx=0x7f3a1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10010630 con 0x7f3a20103e30 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a10010cb0 con 0x7f3a20103e30 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10018bd0 con 0x7f3a20103e30 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2019f720 con 0x7f3a20103e30 2024-09-12T03:05:41.174 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.171+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201a7000 con 0x7f3a20103e30 2024-09-12T03:05:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.172+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20104210 con 0x7f3a20103e30 2024-09-12T03:05:41.175 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.174+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a100107d0 con 0x7f3a20103e30 2024-09-12T03:05:41.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.176+0000 7f3a17e00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 0x7f3a040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:41.176 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.176+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3a100cfa10 con 0x7f3a20103e30 2024-09-12T03:05:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.176+0000 7f3a16000640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 0x7f3a040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:41.177 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.177+0000 7f3a16000640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 0x7f3a040a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009230 tx=0x7f3a080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:41.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.180+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a1009bef0 con 0x7f3a20103e30 2024-09-12T03:05:41.367 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.366+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a2010c200 con 0x7f3a20103e30 2024-09-12T03:05:41.369 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.369+0000 7f3a17e00640 1 -- 172.21.15.57:0/366493067 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24911 (secure 0 0 0) 0x7f3a100a01f0 con 0x7f3a20103e30 2024-09-12T03:05:41.370 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:05:41.370 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":96,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:04:42.450539+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:05:41.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.371+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 msgr2=0x7f3a040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.371+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 0x7f3a040a50c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009230 tx=0x7f3a080058d0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.372 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.371+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 msgr2=0x7f3a2019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.371+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000ebc0 tx=0x7f3a1000c6a0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 shutdown_connections 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3a040a2c00 0x7f3a040a50c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a20108f60 0x7f3a2019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a201047e0 0x7f3a2019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 --2- 172.21.15.57:0/366493067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3a20103e30 0x7f3a2019a480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 >> 172.21.15.57:0/366493067 conn(0x7f3a200ff8d0 msgr2=0x7f3a20100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 shutdown_connections 2024-09-12T03:05:41.373 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:41.372+0000 7f3a1fe00640 1 -- 172.21.15.57:0/366493067 wait complete. 2024-09-12T03:05:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:41 smithi057 ceph-mon[30361]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:41 smithi057 ceph-mon[30361]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T03:05:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:41 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/366493067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:41 smithi204 ceph-mon[35013]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:41 smithi204 ceph-mon[35013]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T03:05:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:41 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/366493067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:41 smithi181 ceph-mon[35320]: pgmap v386: 129 pgs: 129 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:05:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:41 smithi181 ceph-mon[35320]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-12T03:05:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:41 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/366493067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:05:42.179 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:03:08.679612+0000', 'modified': '2024-09-12T03:03:58.893616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24554, 'mds_1': 24548, 'mds_2': 24529}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/1804109241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 1804109241}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1804109241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/368351857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 368351857}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 368351857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.181:6837/2458916124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2458916124}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2458916124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.204:6839/3278241191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 3278241191}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 3278241191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.57:6841/1479422251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.57:6840', 'nonce': 1479422251}, {'type': 'v1', 'addr': '172.21.15.57:6841', 'nonce': 1479422251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24554': {'gid': 24554, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.181:6839/399579254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6838', 'nonce': 399579254}, {'type': 'v1', 'addr': '172.21.15.181:6839', 'nonce': 399579254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24554, 'qdb_cluster': [24529, 24548, 24554]} 2024-09-12T03:05:42.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T03:05:42.180 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:05:42.559 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:43.180 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.179+0000 7f0218400640 1 -- 172.21.15.57:0/2067176750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 msgr2=0x7f02101124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.179+0000 7f0218400640 1 --2- 172.21.15.57:0/2067176750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101124f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009900 tx=0x7f01fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 -- 172.21.15.57:0/2067176750 shutdown_connections 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 --2- 172.21.15.57:0/2067176750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210112a30 0x7f0210114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 --2- 172.21.15.57:0/2067176750 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101124f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 --2- 172.21.15.57:0/2067176750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210076720 0x7f0210076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 -- 172.21.15.57:0/2067176750 >> 172.21.15.57:0/2067176750 conn(0x7f02100ffa90 msgr2=0x7f0210101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 -- 172.21.15.57:0/2067176750 shutdown_connections 2024-09-12T03:05:43.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.180+0000 7f0218400640 1 -- 172.21.15.57:0/2067176750 wait complete. 2024-09-12T03:05:43.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 Processor -- start 2024-09-12T03:05:43.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 -- start start 2024-09-12T03:05:43.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210076720 0x7f02101a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0210118610 con 0x7f0210077040 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0210118490 con 0x7f0210076720 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.181+0000 7f0218400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0210118790 con 0x7f0210112a30 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0215a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210076720 0x7f02101a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60784/0 (socket says 172.21.15.57:60784) 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 -- 172.21.15.57:0/3568642020 learned_addr learned my addr 172.21.15.57:0/3568642020 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0215000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 -- 172.21.15.57:0/3568642020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210076720 msgr2=0x7f02101a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210076720 0x7f02101a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 -- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 msgr2=0x7f02101a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 -- 172.21.15.57:0/3568642020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021019f030 con 0x7f0210112a30 2024-09-12T03:05:43.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.182+0000 7f0216400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f020800dac0 tx=0x7f020800df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.183+0000 7f020e000640 1 -- 172.21.15.57:0/3568642020 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208017030 con 0x7f0210112a30 2024-09-12T03:05:43.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.183+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021019f320 con 0x7f0210112a30 2024-09-12T03:05:43.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.183+0000 7f020e000640 1 -- 172.21.15.57:0/3568642020 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0208003e50 con 0x7f0210112a30 2024-09-12T03:05:43.184 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.183+0000 7f020e000640 1 -- 172.21.15.57:0/3568642020 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208010460 con 0x7f0210112a30 2024-09-12T03:05:43.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.183+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02101ab610 con 0x7f0210112a30 2024-09-12T03:05:43.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.186+0000 7f020e000640 1 -- 172.21.15.57:0/3568642020 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f020801a070 con 0x7f0210112a30 2024-09-12T03:05:43.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.186+0000 7f021afc1740 1 Processor -- start 2024-09-12T03:05:43.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.186+0000 7f021afc1740 1 -- start start 2024-09-12T03:05:43.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.186+0000 7f021afc1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f021afc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604391fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f021afc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x563604394900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f021afc1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5636042ed0b0 con 0x563604387330 2024-09-12T03:05:43.187 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f021afc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563604294ab0 con 0x563604392510 2024-09-12T03:05:43.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f021afc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636042e9070 con 0x563604387c50 2024-09-12T03:05:43.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604391fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48608/0 (socket says 172.21.15.57:48608) 2024-09-12T03:05:43.188 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215000640 1 -- 172.21.15.57:0/1472123964 learned_addr learned my addr 172.21.15.57:0/1472123964 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:43.189 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215000640 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x563604394900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 -- 172.21.15.57:0/1472123964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 msgr2=0x563604394900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x563604394900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 -- 172.21.15.57:0/1472123964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 msgr2=0x563604387710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215a00640 1 -- 172.21.15.57:0/1472123964 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636042a14a0 con 0x563604387c50 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.187+0000 7f0215000640 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f0215a00640 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604391fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0204009bd0 tx=0x7f0204011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdfc0f7106c6ad71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f020e000640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 0x7f01f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f0202a00640 1 -- 172.21.15.57:0/1472123964 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204004230 con 0x563604387c50 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f0202a00640 1 -- 172.21.15.57:0/1472123964 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0204009e40 con 0x563604387c50 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f0202a00640 1 -- 172.21.15.57:0/1472123964 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204004d70 con 0x563604387c50 2024-09-12T03:05:43.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f020e000640 1 -- 172.21.15.57:0/3568642020 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f02080cf150 con 0x7f0210112a30 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.188+0000 7f0215000640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 0x7f01f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 -- 172.21.15.57:0/1472123964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 msgr2=0x563604391fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604391fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0204009bd0 tx=0x7f0204011810 comp rx=0 tx=0).stop 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 -- 172.21.15.57:0/1472123964 shutdown_connections 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x563604394900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604391fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 --2- 172.21.15.57:0/1472123964 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x563604387710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f0215000640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 0x7f01f80a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01fc031550 tx=0x7f01fc005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 -- 172.21.15.57:0/1472123964 >> 172.21.15.57:0/1472123964 conn(0x5636043838c0 msgr2=0x563604383ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 -- 172.21.15.57:0/1472123964 shutdown_connections 2024-09-12T03:05:43.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.189+0000 7f021afc1740 1 -- 172.21.15.57:0/1472123964 wait complete. 2024-09-12T03:05:43.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 -- start start 2024-09-12T03:05:43.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 Processor -- start 2024-09-12T03:05:43.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x56360438f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x56360438fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x563604395060 con 0x563604387330 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x563604275920 con 0x563604392510 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636043951e0 con 0x563604387c50 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.190+0000 7f021afc1740 10 client.0 _ll_register_callbacks cb 0x5636042998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x56360438fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x56360438f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x56360438fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60808/0 (socket says 172.21.15.57:60808) 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215000640 1 -- 172.21.15.57:0/3951396208 learned_addr learned my addr 172.21.15.57:0/3951396208 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0216400640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 -- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 msgr2=0x56360438fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x56360438fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 -- 172.21.15.57:0/3951396208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 msgr2=0x56360438ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.191+0000 7f0215a00640 1 -- 172.21.15.57:0/3951396208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360405e2b0 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f0215a00640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x56360438f530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0204011d10 tx=0x7f0204003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4bb75e0ab962c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f01f7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204027020 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f0204003c10 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636042fa620 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0204032050 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f021afc1740 10 client.15063 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.192+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636043a0df0 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5636042fa620 con 0x563604387330 2024-09-12T03:05:43.195 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0204018070 con 0x563604387330 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 10 client.15063 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 20 client.15063 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x5636043a3260 0x5636043a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 4 client.15063 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:43.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 --> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636042f4b30 con 0x5636043a3260 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.193+0000 7f0215000640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x5636043a3260 0x5636043a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.194+0000 7f0215000640 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x5636043a3260 0x5636043a46d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ad68613c89fbd6f server_cookie=a602e3a07753a55f in_seq=0 out_seq=0 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.194+0000 7f01f7400640 10 client.15063 ms_handle_connect on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f01f7400640 1 -- 172.21.15.57:0/3951396208 <== mds.2 v2:172.21.15.204:6838/3278241191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636042f4b30 con 0x5636043a3260 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f01f7400640 10 client.15063 handle_command_reply: tid=0 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 client.15063 shutdown 2024-09-12T03:05:43.197 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 msgr2=0x56360438f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x56360438f530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0204011d10 tx=0x7f0204003820 comp rx=0 tx=0).stop 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 shutdown_connections 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x5636043a3260 0x5636043a46d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x56360438fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 --2- 172.21.15.57:0/3951396208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x56360438f530 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 >> 172.21.15.57:0/3951396208 conn(0x5636043838c0 msgr2=0x5636043895f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.195+0000 7f01f7400640 0 client.15063 ms_handle_reset on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.196+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 shutdown_connections 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.196+0000 7f021afc1740 1 -- 172.21.15.57:0/3951396208 wait complete. 2024-09-12T03:05:43.198 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.196+0000 7f021afc1740 20 client.15063 trim_cache size 0 max 16384 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 Processor -- start 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 -- start start 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x5636043a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604389d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5636043a2a10 con 0x563604387330 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636043a30e0 con 0x563604392510 2024-09-12T03:05:43.216 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.215+0000 7f021afc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636043a37b0 con 0x563604387c50 2024-09-12T03:05:43.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0215000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604389d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0215a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x5636043a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0215000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604389d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60816/0 (socket says 172.21.15.57:60816) 2024-09-12T03:05:43.217 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58668/0 (socket says 172.21.15.57:58668) 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0215000640 1 -- 172.21.15.57:0/3862912205 learned_addr learned my addr 172.21.15.57:0/3862912205 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 -- 172.21.15.57:0/3862912205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 msgr2=0x563604389d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604389d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 -- 172.21.15.57:0/3862912205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 msgr2=0x5636043a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x5636043a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 -- 172.21.15.57:0/3862912205 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360438dac0 con 0x563604392510 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0216400640 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f020809b280 tx=0x7f020809bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae807c2230e472b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.216+0000 7f0202000640 1 -- 172.21.15.57:0/3862912205 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02080a41f0 con 0x563604392510 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f0202000640 1 -- 172.21.15.57:0/3862912205 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0208099620 con 0x563604392510 2024-09-12T03:05:43.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f0202000640 1 -- 172.21.15.57:0/3862912205 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208097d80 con 0x563604392510 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 -- 172.21.15.57:0/3862912205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 msgr2=0x56360438fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f020809b280 tx=0x7f020809bc80 comp rx=0 tx=0).stop 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 -- 172.21.15.57:0/3862912205 shutdown_connections 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x56360438fe00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387c50 0x563604389d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 --2- 172.21.15.57:0/3862912205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387330 0x5636043a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.220 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 -- 172.21.15.57:0/3862912205 >> 172.21.15.57:0/3862912205 conn(0x5636043838c0 msgr2=0x56360438de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.217+0000 7f021afc1740 1 -- 172.21.15.57:0/3862912205 shutdown_connections 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.218+0000 7f021afc1740 1 -- 172.21.15.57:0/3862912205 wait complete. 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.218+0000 7f021afc1740 1 -- start start 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.218+0000 7f021afc1740 1 Processor -- start 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387330 0x56360438bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x5636043ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5636043a4550 con 0x563604387c50 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5636043a3e80 con 0x563604392510 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5636043a4c20 con 0x563604387330 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f021afc1740 10 client.0 _ll_register_callbacks cb 0x7f0204004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48638/0 (socket says 172.21.15.57:48638) 2024-09-12T03:05:43.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 -- 172.21.15.57:0/4157681710 learned_addr learned my addr 172.21.15.57:0/4157681710 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0216400640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x5636043ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215a00640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387330 0x56360438bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 -- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387330 msgr2=0x56360438bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387330 0x56360438bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 -- 172.21.15.57:0/4157681710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 msgr2=0x5636043ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x5636043ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0215000640 1 -- 172.21.15.57:0/4157681710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636043abdf0 con 0x563604387c50 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.219+0000 7f0216400640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x5636043ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f0215000640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0044e0 tx=0x7f01fc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a9dd8f457b855d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:43.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f01f7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:05:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc035040 con 0x563604387c50 2024-09-12T03:05:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f01fc045070 con 0x563604387c50 2024-09-12T03:05:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0208099620 con 0x563604387c50 2024-09-12T03:05:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01fc067050 con 0x563604387c50 2024-09-12T03:05:43.224 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f021afc1740 10 client.15069 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.220+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5636043ad160 con 0x563604387c50 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0208099620 con 0x563604387c50 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f01fc048040 con 0x563604387c50 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 20 client.15069 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x563604411e80 0x563604410b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 4 client.15069 mds_command: new command op to 24529 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 --> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] -- command(tid 0: {"prefix": "status"}) -- 0x563604270f80 con 0x563604411e80 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.221+0000 7f0215a00640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x563604411e80 0x563604410b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f0215a00640 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x563604411e80 0x563604410b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f29abe2b502f55d server_cookie=916d5f770179de1d in_seq=0 out_seq=0 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f01f7400640 10 client.15069 ms_handle_connect on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f01f7400640 1 -- 172.21.15.57:0/4157681710 <== mds.2 v2:172.21.15.204:6838/3278241191 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563604270f80 con 0x563604411e80 2024-09-12T03:05:43.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f01f7400640 10 client.15069 handle_command_reply: tid=0 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 client.15069 shutdown 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 msgr2=0x56360438c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0044e0 tx=0x7f01fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 shutdown_connections 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:6838/3278241191,v1:172.21.15.204:6839/3278241191] conn(0x563604411e80 0x563604410b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563604392510 0x5636043ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x563604387c50 0x56360438c4c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 --2- 172.21.15.57:0/4157681710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563604387330 0x56360438bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 >> 172.21.15.57:0/4157681710 conn(0x5636043838c0 msgr2=0x56360439d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.222+0000 7f01f7400640 0 client.15069 ms_handle_reset on v2:172.21.15.204:6838/3278241191 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 shutdown_connections 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f021afc1740 1 -- 172.21.15.57:0/4157681710 wait complete. 2024-09-12T03:05:43.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f021afc1740 20 client.15069 trim_cache size 0 max 16384 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 2, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24529, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 107.423447995, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:43.227 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 177.14685379700001 2024-09-12T03:05:43.228 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:43.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 msgr2=0x7f01f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 0x7f01f80a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f01fc031550 tx=0x7f01fc005bc0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 msgr2=0x7f021019e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:43.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.223+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f020800dac0 tx=0x7f020800df80 comp rx=0 tx=0).stop 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 shutdown_connections 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f01f80a2b80 0x7f01f80a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0210112a30 0x7f021019e900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0210077040 0x7f02101a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 --2- 172.21.15.57:0/3568642020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0210076720 0x7f02101a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 >> 172.21.15.57:0/3568642020 conn(0x7f02100ffa90 msgr2=0x7f0210101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 shutdown_connections 2024-09-12T03:05:43.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:43.224+0000 7f0218400640 1 -- 172.21.15.57:0/3568642020 wait complete. 2024-09-12T03:05:43.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:43 smithi204 ceph-mon[35013]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:05:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:43 smithi181 ceph-mon[35320]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:05:43.719 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:43 smithi057 ceph-mon[30361]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-12T03:05:44.199 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-12T03:05:44.576 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:45.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 -- 172.21.15.57:0/4214138693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 msgr2=0x7f2fe8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 --2- 172.21.15.57:0/4214138693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe8110cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009ca0 tx=0x7f2fe002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 -- 172.21.15.57:0/4214138693 shutdown_connections 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 --2- 172.21.15.57:0/4214138693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe8110cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 --2- 172.21.15.57:0/4214138693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe81049c0 0x7f2fe8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 --2- 172.21.15.57:0/4214138693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe8104010 0x7f2fe81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 -- 172.21.15.57:0/4214138693 >> 172.21.15.57:0/4214138693 conn(0x7f2fe80ffa50 msgr2=0x7f2fe8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 -- 172.21.15.57:0/4214138693 shutdown_connections 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.146+0000 7f2fefc00640 1 -- 172.21.15.57:0/4214138693 wait complete. 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.147+0000 7f2fefc00640 1 Processor -- start 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.147+0000 7f2fefc00640 1 -- start start 2024-09-12T03:05:45.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 0x7f2fe819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 0x7f2fe819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81141a0 con 0x7f2fe8109140 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8114020 con 0x7f2fe8104010 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fefc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8114320 con 0x7f2fe81049c0 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48652/0 (socket says 172.21.15.57:48652) 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 -- 172.21.15.57:0/3504704598 learned_addr learned my addr 172.21.15.57:0/3504704598 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fed200640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 0x7f2fe819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 0x7f2fe819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 -- 172.21.15.57:0/3504704598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 msgr2=0x7f2fe819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 0x7f2fe819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 -- 172.21.15.57:0/3504704598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 msgr2=0x7f2fe819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.148+0000 7f2fedc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 0x7f2fe819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fedc00640 1 -- 172.21.15.57:0/3504704598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe819f650 con 0x7f2fe8109140 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 0x7f2fe819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fed200640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 0x7f2fe819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fedc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005e30 tx=0x7f2fe0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fe6000640 1 -- 172.21.15.57:0/3504704598 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0031650 con 0x7f2fe8109140 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fe6000640 1 -- 172.21.15.57:0/3504704598 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe00317f0 con 0x7f2fe8109140 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fe6000640 1 -- 172.21.15.57:0/3504704598 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe002f9d0 con 0x7f2fe8109140 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe819f8e0 con 0x7f2fe8109140 2024-09-12T03:05:45.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.149+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe81a7220 con 0x7f2fe8109140 2024-09-12T03:05:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.152+0000 7f2ff27f2740 1 Processor -- start 2024-09-12T03:05:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.152+0000 7f2fe6000640 1 -- 172.21.15.57:0/3504704598 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2fe002fbf0 con 0x7f2fe8109140 2024-09-12T03:05:45.153 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 -- start start 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b8bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b8c510 0x55e8f5b8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5ae70b0 con 0x55e8f5b81330 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5a8eab0 con 0x55e8f5b81c50 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5ae3070 con 0x55e8f5b8c510 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48658/0 (socket says 172.21.15.57:48658) 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2fdfe00640 1 -- 172.21.15.57:0/499641379 learned_addr learned my addr 172.21.15.57:0/499641379 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:45.154 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.153+0000 7f2fed200640 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b8bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.155 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b8c510 0x55e8f5b8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 -- 172.21.15.57:0/499641379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b8c510 msgr2=0x55e8f5b8e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b8c510 0x55e8f5b8e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 -- 172.21.15.57:0/499641379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 msgr2=0x55e8f5b8bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b8bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 -- 172.21.15.57:0/499641379 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f5a9b4a0 con 0x55e8f5b81330 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b7ec40 tx=0x7f2fd0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b750c96b547fb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.156 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fe6000640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 0x7f2fcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.154+0000 7f2fde000640 1 -- 172.21.15.57:0/499641379 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0013170 con 0x55e8f5b81330 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2fe6000640 1 -- 172.21.15.57:0/3504704598 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2fe00f0790 con 0x7f2fe8109140 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2fde000640 1 -- 172.21.15.57:0/499641379 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd0005500 con 0x55e8f5b81330 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2fde000640 1 -- 172.21.15.57:0/499641379 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004a00 con 0x55e8f5b81330 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2fed200640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 0x7f2fcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2fed200640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 0x7f2fcc0a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b7e720 tx=0x7f2fd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 -- 172.21.15.57:0/499641379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 msgr2=0x55e8f5b81710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b7ec40 tx=0x7f2fd0005850 comp rx=0 tx=0).stop 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 -- 172.21.15.57:0/499641379 shutdown_connections 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b8c510 0x55e8f5b8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b8bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/499641379 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81330 0x55e8f5b81710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.155+0000 7f2ff27f2740 1 -- 172.21.15.57:0/499641379 >> 172.21.15.57:0/499641379 conn(0x55e8f5b7d8c0 msgr2=0x55e8f5b7dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.156+0000 7f2ff27f2740 1 -- 172.21.15.57:0/499641379 shutdown_connections 2024-09-12T03:05:45.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.156+0000 7f2ff27f2740 1 -- 172.21.15.57:0/499641379 wait complete. 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.156+0000 7f2ff27f2740 1 -- start start 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.156+0000 7f2ff27f2740 1 Processor -- start 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b89550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 0x55e8f5b89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b8f060 con 0x55e8f5b8c510 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5a6f920 con 0x55e8f5b81c50 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b8f1e0 con 0x55e8f5b81330 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2ff27f2740 10 client.0 _ll_register_callbacks cb 0x55e8f5a938a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 0x55e8f5b89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2fed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b89550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2fdfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 0x55e8f5b89fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48672/0 (socket says 172.21.15.57:48672) 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.157+0000 7f2fedc00640 1 -- 172.21.15.57:0/822999702 learned_addr learned my addr 172.21.15.57:0/822999702 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 -- 172.21.15.57:0/822999702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 msgr2=0x55e8f5b89550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b89550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 -- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 msgr2=0x55e8f5b89fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 0x55e8f5b89fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 -- 172.21.15.57:0/822999702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f58582b0 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0012040 tx=0x7f2fd0013e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a445958317b90af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0025070 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fd0005500 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.158+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2fd00046c0 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2fd0031050 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2ff27f2740 10 client.25124 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8f5b9add0 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fd0005500 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2fd00206a0 con 0x55e8f5b81c50 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.159+0000 7f2ff27f2740 10 client.25124 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2ff27f2740 10 client.25124 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2ff27f2740 20 client.25124 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:45.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b9d120 0x55e8f5b9e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2ff27f2740 4 client.25124 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:45.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 --> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8f5af4620 con 0x55e8f5b9d120 2024-09-12T03:05:45.163 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2fed200640 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b9d120 0x55e8f5b9e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2fed200640 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b9d120 0x55e8f5b9e590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf45f4c6967d67ea server_cookie=33a54143fe2b61ee in_seq=0 out_seq=0 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.160+0000 7f2fd6a00640 10 client.25124 ms_handle_connect on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.161+0000 7f2fd6a00640 1 -- 172.21.15.57:0/822999702 <== mds.1 v2:172.21.15.57:6840/1479422251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8f5af4620 con 0x55e8f5b9d120 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.161+0000 7f2fd6a00640 10 client.25124 handle_command_reply: tid=0 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.161+0000 7f2ff27f2740 1 client.25124 shutdown 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.161+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 msgr2=0x55e8f5b89a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.161+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89a90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0012040 tx=0x7f2fd0013e70 comp rx=0 tx=0).stop 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 shutdown_connections 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b9d120 0x55e8f5b9e590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b8c510 0x55e8f5b89fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2fd6a00640 0 client.25124 ms_handle_reset on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89a90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/822999702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b89550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 >> 172.21.15.57:0/822999702 conn(0x55e8f5b7d8c0 msgr2=0x55e8f5b835f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 shutdown_connections 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 1 -- 172.21.15.57:0/822999702 wait complete. 2024-09-12T03:05:45.165 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.162+0000 7f2ff27f2740 20 client.25124 trim_cache size 0 max 16384 2024-09-12T03:05:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 Processor -- start 2024-09-12T03:05:45.181 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 -- start start 2024-09-12T03:05:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5b7d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b8b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.182 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9c8d0 con 0x55e8f5b81c50 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9cfa0 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9d670 con 0x55e8f5b81330 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48684/0 (socket says 172.21.15.57:48684) 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.181+0000 7f2fedc00640 1 -- 172.21.15.57:0/1518032743 learned_addr learned my addr 172.21.15.57:0/1518032743 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b8b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fed200640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5b7d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 -- 172.21.15.57:0/1518032743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 msgr2=0x55e8f5b8b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b8b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 -- 172.21.15.57:0/1518032743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b7d8c0 msgr2=0x55e8f5b7d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5b7d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 -- 172.21.15.57:0/1518032743 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8f5b7e970 con 0x55e8f5b81c50 2024-09-12T03:05:45.183 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b8b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fedc00640 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009a60 tx=0x7f2fe00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9da1bd6d54ee392 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.182+0000 7f2fdd600640 1 -- 172.21.15.57:0/1518032743 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00da070 con 0x55e8f5b81c50 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2fdd600640 1 -- 172.21.15.57:0/1518032743 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0036070 con 0x55e8f5b81c50 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2fdd600640 1 -- 172.21.15.57:0/1518032743 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00f0410 con 0x55e8f5b81c50 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 -- 172.21.15.57:0/1518032743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 msgr2=0x55e8f5b89e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009a60 tx=0x7f2fe00093d0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 -- 172.21.15.57:0/1518032743 shutdown_connections 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5b89e20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.185 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b81330 0x55e8f5b8b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/1518032743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5b7d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 -- 172.21.15.57:0/1518032743 >> 172.21.15.57:0/1518032743 conn(0x55e8f5b9c4f0 msgr2=0x55e8f5b8adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 -- 172.21.15.57:0/1518032743 shutdown_connections 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.183+0000 7f2ff27f2740 1 -- 172.21.15.57:0/1518032743 wait complete. 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.184+0000 7f2ff27f2740 1 -- start start 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.184+0000 7f2ff27f2740 1 Processor -- start 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.184+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81330 0x55e8f5ba0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5ba1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9e410 con 0x55e8f5b81c50 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9dd40 con 0x55e8f5b81330 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e8f5b9eae0 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.186 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2ff27f2740 10 client.0 _ll_register_callbacks cb 0x7f2fd0013a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:45.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5ba1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.190 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fedc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5ba1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48700/0 (socket says 172.21.15.57:48700) 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60876/0 (socket says 172.21.15.57:60876) 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fedc00640 1 -- 172.21.15.57:0/3361813781 learned_addr learned my addr 172.21.15.57:0/3361813781 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81330 0x55e8f5ba0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 -- 172.21.15.57:0/3361813781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81330 msgr2=0x55e8f5ba0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81330 0x55e8f5ba0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 -- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 msgr2=0x55e8f5ba1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5ba1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.185+0000 7f2fed200640 1 -- 172.21.15.57:0/3361813781 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8f5ba4e30 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2fed200640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b9f3d0 tx=0x7f2fd8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7dfbc0123e4ed54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2fd6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:45.191 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8004020 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe0036070 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f2fd80041c0 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2fd80043c0 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2ff27f2740 10 client.25138 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.186+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8f5c07c10 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0036070 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2fd8003a70 con 0x55e8f5b7d8c0 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 10 client.25138 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 10 client.25138 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 20 client.25138 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b8c510 0x55e8f5c0b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 4 client.25138 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 --> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] -- command(tid 0: {"prefix": "status"}) -- 0x55e8f5aeeb30 con 0x55e8f5b8c510 2024-09-12T03:05:45.192 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.187+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b8c510 0x55e8f5c0b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2fdfe00640 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b8c510 0x55e8f5c0b520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2d7998705e53aa9 server_cookie=494c52023e080c5c in_seq=0 out_seq=0 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2fd6a00640 10 client.25138 ms_handle_connect on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2fd6a00640 1 -- 172.21.15.57:0/3361813781 <== mds.1 v2:172.21.15.57:6840/1479422251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55e8f5aeeb30 con 0x55e8f5b8c510 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2fd6a00640 10 client.25138 handle_command_reply: tid=0 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 client.25138 shutdown 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 msgr2=0x55e8f5ba05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b9f3d0 tx=0x7f2fd8017000 comp rx=0 tx=0).stop 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 shutdown_connections 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:6840/1479422251,v1:172.21.15.57:6841/1479422251] conn(0x55e8f5b8c510 0x55e8f5c0b520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e8f5b81c50 0x55e8f5ba1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e8f5b81330 0x55e8f5ba0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 --2- 172.21.15.57:0/3361813781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e8f5b7d8c0 0x55e8f5ba05c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.193 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2fd6a00640 0 client.25138 ms_handle_reset on v2:172.21.15.57:6840/1479422251 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.188+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 >> 172.21.15.57:0/3361813781 conn(0x55e8f5b9c4f0 msgr2=0x55e8f5b83530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 shutdown_connections 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2ff27f2740 1 -- 172.21.15.57:0/3361813781 wait complete. 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2ff27f2740 20 client.25138 trim_cache size 0 max 16384 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 1, 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24548, 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:45.194 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 110.403187594, 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 170.92914105 2024-09-12T03:05:45.195 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 msgr2=0x7f2fcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 0x7f2fcc0a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e8f5b7e720 tx=0x7f2fd8005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 msgr2=0x7f2fe819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005e30 tx=0x7f2fe0003d00 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 shutdown_connections 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f2fcc0a2b80 0x7f2fcc0a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2fe8109140 0x7f2fe819ef70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fe81049c0 0x7f2fe819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 --2- 172.21.15.57:0/3504704598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe8104010 0x7f2fe819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.189+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 >> 172.21.15.57:0/3504704598 conn(0x7f2fe80ffa50 msgr2=0x7f2fe8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.190+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 shutdown_connections 2024-09-12T03:05:45.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:45.190+0000 7f2fefc00640 1 -- 172.21.15.57:0/3504704598 wait complete. 2024-09-12T03:05:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:45 smithi204 ceph-mon[35013]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:05:45.646 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:45 smithi057 ceph-mon[30361]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:05:45.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:45 smithi181 ceph-mon[35320]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T03:05:46.161 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:05:46.538 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 -- 172.21.15.57:0/523942211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb81034b0 msgr2=0x7f1cb810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 --2- 172.21.15.57:0/523942211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb81034b0 0x7f1cb810b9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009860 tx=0x7f1ca002a3d0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 -- 172.21.15.57:0/523942211 shutdown_connections 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 --2- 172.21.15.57:0/523942211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb810bfc0 0x7f1cb810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 --2- 172.21.15.57:0/523942211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb81034b0 0x7f1cb810b9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 --2- 172.21.15.57:0/523942211 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 -- 172.21.15.57:0/523942211 >> 172.21.15.57:0/523942211 conn(0x7f1cb80fc610 msgr2=0x7f1cb80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 -- 172.21.15.57:0/523942211 shutdown_connections 2024-09-12T03:05:47.114 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.113+0000 7f1cbe600640 1 -- 172.21.15.57:0/523942211 wait complete. 2024-09-12T03:05:47.115 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 Processor -- start 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 -- start start 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb81034b0 0x7f1cb819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb810bfc0 0x7f1cb819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111dc0 con 0x7f1cb8102b90 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111c40 con 0x7f1cb810bfc0 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.114+0000 7f1cbe600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8111f40 con 0x7f1cb81034b0 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb81034b0 0x7f1cb819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48718/0 (socket says 172.21.15.57:48718) 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 -- 172.21.15.57:0/3003530246 learned_addr learned my addr 172.21.15.57:0/3003530246 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb810bfc0 0x7f1cb819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.116 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 -- 172.21.15.57:0/3003530246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb81034b0 msgr2=0x7f1cb819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb81034b0 0x7f1cb819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 -- 172.21.15.57:0/3003530246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb810bfc0 msgr2=0x7f1cb819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb810bfc0 0x7f1cb819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 -- 172.21.15.57:0/3003530246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb819f490 con 0x7f1cb8102b90 2024-09-12T03:05:47.117 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.115+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800e9c0 tx=0x7f1ca800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.116+0000 7f1cb6000640 1 -- 172.21.15.57:0/3003530246 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800cde0 con 0x7f1cb8102b90 2024-09-12T03:05:47.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.116+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb819f780 con 0x7f1cb8102b90 2024-09-12T03:05:47.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.116+0000 7f1cb6000640 1 -- 172.21.15.57:0/3003530246 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca80035f0 con 0x7f1cb8102b90 2024-09-12T03:05:47.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.116+0000 7f1cb6000640 1 -- 172.21.15.57:0/3003530246 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8010690 con 0x7f1cb8102b90 2024-09-12T03:05:47.118 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.116+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81a7060 con 0x7f1cb8102b90 2024-09-12T03:05:47.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.119+0000 7f1cb6000640 1 -- 172.21.15.57:0/3003530246 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ca80037a0 con 0x7f1cb8102b90 2024-09-12T03:05:47.119 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.119+0000 7f1cc25c2740 1 Processor -- start 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.119+0000 7f1cc25c2740 1 -- start start 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0eca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0e300b0 con 0x55c6c0ed56c0 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0dd7ab0 con 0x55c6c0eca4e0 2024-09-12T03:05:47.120 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0e2c070 con 0x55c6c0ecae00 2024-09-12T03:05:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48730/0 (socket says 172.21.15.57:48730) 2024-09-12T03:05:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 -- 172.21.15.57:0/804170081 learned_addr learned my addr 172.21.15.57:0/804170081 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:47.121 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0eca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 -- 172.21.15.57:0/804170081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 msgr2=0x55c6c0ed5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 -- 172.21.15.57:0/804170081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 msgr2=0x55c6c0eca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0eca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.120+0000 7f1cafe00640 1 -- 172.21.15.57:0/804170081 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c0de44a0 con 0x55c6c0ed56c0 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cb6000640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 0x7f1c9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cb6000640 1 -- 172.21.15.57:0/3003530246 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1ca80cf6f0 con 0x7f1cb8102b90 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 0x7f1c9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.122 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cafe00640 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009860 tx=0x7f1ca0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f311e19469a7394 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cae000640 1 -- 172.21.15.57:0/804170081 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0030cb0 con 0x55c6c0ed56c0 2024-09-12T03:05:47.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cae000640 1 -- 172.21.15.57:0/804170081 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca0030e50 con 0x55c6c0ed56c0 2024-09-12T03:05:47.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.121+0000 7f1cae000640 1 -- 172.21.15.57:0/804170081 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004920 con 0x55c6c0ed56c0 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 -- 172.21.15.57:0/804170081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 msgr2=0x55c6c0ed7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 0x7f1c9c0a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005fd0 tx=0x7f1cb0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009860 tx=0x7f1ca0032f60 comp rx=0 tx=0).stop 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 -- 172.21.15.57:0/804170081 shutdown_connections 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed7ab0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/804170081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0eca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 -- 172.21.15.57:0/804170081 >> 172.21.15.57:0/804170081 conn(0x55c6c0ec6a70 msgr2=0x55c6c0ec6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 -- 172.21.15.57:0/804170081 shutdown_connections 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.122+0000 7f1cc25c2740 1 -- 172.21.15.57:0/804170081 wait complete. 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 -- start start 2024-09-12T03:05:47.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 Processor -- start 2024-09-12T03:05:47.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.126 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.123+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ed8330 con 0x55c6c0eca4e0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0db8920 con 0x55c6c0ed56c0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ed84b0 con 0x55c6c0ecae00 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cc25c2740 10 client.0 _ll_register_callbacks cb 0x55c6c0ddc8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48738/0 (socket says 172.21.15.57:48738) 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbd200640 1 -- 172.21.15.57:0/2905895575 learned_addr learned my addr 172.21.15.57:0/2905895575 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 -- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 msgr2=0x55c6c0ed3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 -- 172.21.15.57:0/2905895575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 msgr2=0x55c6c0ed2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.127 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cafe00640 1 -- 172.21.15.57:0/2905895575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c0ba12b0 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.124+0000 7f1cbd200640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1cafe00640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed2b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0032ff0 tx=0x7f1ca0005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188be93eb0cd45c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca004e020 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca0030e50 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1ca00307b0 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ca0059050 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1cc25c2740 10 client.25150 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.125+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6c0ed8330 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca0030e50 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ca0044020 con 0x55c6c0ecae00 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 10 client.25150 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:47.128 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 10 client.25150 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:05:47.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 20 client.25150 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:47.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0ee61d0 0x55c6c0ee7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 4 client.25150 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:05:47.129 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.126+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c0e3d620 con 0x55c6c0ee61d0 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.127+0000 7f1cbd200640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0ee61d0 0x55c6c0ee7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.127+0000 7f1cbd200640 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0ee61d0 0x55c6c0ee7640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f066562fdd9bc7 server_cookie=447bd7852b16a84a in_seq=0 out_seq=0 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.127+0000 7f1ca6a00640 10 client.25150 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1ca6a00640 1 -- 172.21.15.57:0/2905895575 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6c0e3d620 con 0x55c6c0ee61d0 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1ca6a00640 10 client.25150 handle_command_reply: tid=0 2024-09-12T03:05:47.130 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 client.25150 shutdown 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 msgr2=0x55c6c0ed2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed2b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0032ff0 tx=0x7f1ca0005d20 comp rx=0 tx=0).stop 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 shutdown_connections 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0ee61d0 0x55c6c0ee7640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ed3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ed2b40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/2905895575 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ed2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 >> 172.21.15.57:0/2905895575 conn(0x55c6c0ec6a70 msgr2=0x55c6c0ecc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1ca6a00640 0 client.25150 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 shutdown_connections 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.128+0000 7f1cc25c2740 1 -- 172.21.15.57:0/2905895575 wait complete. 2024-09-12T03:05:47.131 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.129+0000 7f1cc25c2740 20 client.25150 trim_cache size 0 max 16384 2024-09-12T03:05:47.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.147+0000 7f1cc25c2740 1 Processor -- start 2024-09-12T03:05:47.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.147+0000 7f1cc25c2740 1 -- start start 2024-09-12T03:05:47.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ee5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee5980 con 0x55c6c0ed56c0 2024-09-12T03:05:47.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee6050 con 0x55c6c0eca4e0 2024-09-12T03:05:47.149 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee6720 con 0x55c6c0ecae00 2024-09-12T03:05:47.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cbdc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cbdc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:48740/0 (socket says 172.21.15.57:48740) 2024-09-12T03:05:47.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cbdc00640 1 -- 172.21.15.57:0/1362625266 learned_addr learned my addr 172.21.15.57:0/1362625266 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:47.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cafe00640 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.150 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.148+0000 7f1cbd200640 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ee5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 -- 172.21.15.57:0/1362625266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 msgr2=0x55c6c0ee61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 -- 172.21.15.57:0/1362625266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 msgr2=0x55c6c0ee5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ee5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 -- 172.21.15.57:0/1362625266 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c0ecac30 con 0x55c6c0ed56c0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00042e0 tx=0x7f1cb001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be84a995f52397f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cad600640 1 -- 172.21.15.57:0/1362625266 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0004b20 con 0x55c6c0ed56c0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cad600640 1 -- 172.21.15.57:0/1362625266 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0004cc0 con 0x55c6c0ed56c0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.149+0000 7f1cad600640 1 -- 172.21.15.57:0/1362625266 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0005810 con 0x55c6c0ed56c0 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- 172.21.15.57:0/1362625266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 msgr2=0x55c6c0ee1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00042e0 tx=0x7f1cb001a000 comp rx=0 tx=0).stop 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- 172.21.15.57:0/1362625266 shutdown_connections 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee1910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/1362625266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ee5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- 172.21.15.57:0/1362625266 >> 172.21.15.57:0/1362625266 conn(0x55c6c0ec6a70 msgr2=0x55c6c0ec17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- 172.21.15.57:0/1362625266 shutdown_connections 2024-09-12T03:05:47.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- 172.21.15.57:0/1362625266 wait complete. 2024-09-12T03:05:47.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.150+0000 7f1cc25c2740 1 -- start start 2024-09-12T03:05:47.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 Processor -- start 2024-09-12T03:05:47.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee74c0 con 0x55c6c0ecae00 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee6df0 con 0x55c6c0ed56c0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6c0ee7b90 con 0x55c6c0eca4e0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cc25c2740 10 client.0 _ll_register_callbacks cb 0x7f1cb00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60938/0 (socket says 172.21.15.57:60938) 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cbd200640 1 -- 172.21.15.57:0/3416581285 learned_addr learned my addr 172.21.15.57:0/3416581285 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.151+0000 7f1cbdc00640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.158 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 -- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 msgr2=0x55c6c0ee8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 -- 172.21.15.57:0/3416581285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ecae00 msgr2=0x55c6c0ee8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 -- 172.21.15.57:0/3416581285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c0ee95d0 con 0x55c6c0eca4e0 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cbd200640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80984a0 tx=0x7f1ca80d5220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa98572944ded70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:47.159 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1ca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca809b480 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb0004cc0 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f1ca8003b10 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ca809c220 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cc25c2740 10 client.25156 fetch_fsmap learned FSMap version 25 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.152+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c6c0f4f9c0 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0004cc0 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ca80ba290 con 0x55c6c0eca4e0 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 10 client.25156 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 10 client.25156 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 20 client.25156 populate_metadata read hostname 'smithi057' 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0f54460 0x55c6c0f53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:47.160 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 4 client.25156 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "status"}) -- 0x55c6c0e37b30 con 0x55c6c0f54460 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.153+0000 7f1cafe00640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0f54460 0x55c6c0f53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cafe00640 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0f54460 0x55c6c0f53170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3938fc543398ebec server_cookie=e7088c96286d247a in_seq=0 out_seq=0 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1ca6a00640 10 client.25156 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1ca6a00640 1 -- 172.21.15.57:0/3416581285 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c6c0e37b30 con 0x55c6c0f54460 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1ca6a00640 10 client.25156 handle_command_reply: tid=0 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cc25c2740 1 client.25156 shutdown 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 msgr2=0x55c6c0ef0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80984a0 tx=0x7f1ca80d5220 comp rx=0 tx=0).stop 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 shutdown_connections 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.154+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c6c0f54460 0x55c6c0f53170 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6c0ed56c0 0x55c6c0ee8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6c0ecae00 0x55c6c0ee8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.161 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 --2- 172.21.15.57:0/3416581285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6c0eca4e0 0x55c6c0ef0080 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1ca6a00640 0 client.25156 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 >> 172.21.15.57:0/3416581285 conn(0x55c6c0ec6a70 msgr2=0x55c6c0ee51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 shutdown_connections 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 1 -- 172.21.15.57:0/3416581285 wait complete. 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cc25c2740 20 client.25156 trim_cache size 0 max 16384 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout: "cluster_fsid": "3c25d9c4-70b2-11ef-bcea-c7b262605968", 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout: "whoami": 0, 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout: "id": 24554, 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout: "want_state": "up:active", 2024-09-12T03:05:47.162 INFO:teuthology.orchestra.run.smithi057.stdout: "state": "up:active", 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "fs_name": "cephfs", 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "rank_uptime": 157.70367533699999, 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "mdsmap_epoch": 25, 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch": 96, 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_epoch_barrier": 94, 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout: "uptime": 170.30473586400001 2024-09-12T03:05:47.163 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 msgr2=0x7f1c9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 0x7f1c9c0a50c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0005fd0 tx=0x7f1cb0009500 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 msgr2=0x7f1cb819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.155+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800e9c0 tx=0x7f1ca800ee80 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 shutdown_connections 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f1c9c0a2c00 0x7f1c9c0a50c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb810bfc0 0x7f1cb819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb81034b0 0x7f1cb819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 --2- 172.21.15.57:0/3003530246 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1cb8102b90 0x7f1cb819a4e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 >> 172.21.15.57:0/3003530246 conn(0x7f1cb80fc610 msgr2=0x7f1cb810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 shutdown_connections 2024-09-12T03:05:47.164 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:47.156+0000 7f1cbe600640 1 -- 172.21.15.57:0/3003530246 wait complete. 2024-09-12T03:05:47.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:47 smithi204 ceph-mon[35013]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:47 smithi181 ceph-mon[35320]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:47.837 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:47 smithi057 ceph-mon[30361]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:48.111 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:05:48.126 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:05:48.126 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:05:48.127 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:05:48.127 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T03:05:48.128 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:05:48.128 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:05:48.128 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T03:05:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 -- 172.21.15.57:0/946162408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8104f50 msgr2=0x7f86b81053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 --2- 172.21.15.57:0/946162408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8104f50 0x7f86b81053d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009900 tx=0x7f86a402a410 comp rx=0 tx=0).stop 2024-09-12T03:05:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 -- 172.21.15.57:0/946162408 shutdown_connections 2024-09-12T03:05:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 --2- 172.21.15.57:0/946162408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8105910 0x7f86b810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:48.286 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 --2- 172.21.15.57:0/946162408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8104f50 0x7f86b81053d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 --2- 172.21.15.57:0/946162408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8103d60 0x7f86b8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 -- 172.21.15.57:0/946162408 >> 172.21.15.57:0/946162408 conn(0x7f86b80ff4e0 msgr2=0x7f86b8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 -- 172.21.15.57:0/946162408 shutdown_connections 2024-09-12T03:05:48.287 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.285+0000 7f86c0400640 1 -- 172.21.15.57:0/946162408 wait complete. 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.286+0000 7f86c0400640 1 Processor -- start 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.286+0000 7f86c0400640 1 -- start start 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8103d60 0x7f86b8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 0x7f86b819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:48.289 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f86b810fa30 con 0x7f86b8105910 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f86b810f8b0 con 0x7f86b8104f50 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86c0400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86b810fbb0 con 0x7f86b8103d60 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55188/0 (socket says 172.21.15.57:55188) 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 -- 172.21.15.57:0/3413672783 learned_addr learned my addr 172.21.15.57:0/3413672783 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86bda00640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8103d60 0x7f86b8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86bd000640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 0x7f86b819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 -- 172.21.15.57:0/3413672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8103d60 msgr2=0x7f86b8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8103d60 0x7f86b8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 -- 172.21.15.57:0/3413672783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 msgr2=0x7f86b819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 0x7f86b819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86be400640 1 -- 172.21.15.57:0/3413672783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b81a1bc0 con 0x7f86b8105910 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.287+0000 7f86bd000640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 0x7f86b819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:48.290 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86be400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f86b000eb10 tx=0x7f86b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0004b20 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b0004cc0 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b81a1eb0 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b0010760 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.288+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b81a2370 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.289+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b810b080 con 0x7f86b8105910 2024-09-12T03:05:48.291 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.291+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86b0005020 con 0x7f86b8105910 2024-09-12T03:05:48.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.292+0000 7f86b6000640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 0x7f86a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:48.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.293+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f86b00cfbc0 con 0x7f86b8105910 2024-09-12T03:05:48.293 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.293+0000 7f86bda00640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 0x7f86a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:48.294 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.294+0000 7f86bda00640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 0x7f86a00a50f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86b8104db0 tx=0x7f86ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:48.297 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.297+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86b009c6f0 con 0x7f86b8105910 2024-09-12T03:05:48.533 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:48.533+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f86b8062e90 con 0x7f86b8105910 2024-09-12T03:05:49.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.407+0000 7f86b6000640 1 -- 172.21.15.57:0/3413672783 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f86b009bff0 con 0x7f86b8105910 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 msgr2=0x7f86a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 0x7f86a00a50f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f86b8104db0 tx=0x7f86ac005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 msgr2=0x7f86b81a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f86b000eb10 tx=0x7f86b000c2d0 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 shutdown_connections 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f86a00a2c30 0x7f86a00a50f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f86b8105910 0x7f86b81a1550 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86b8104f50 0x7f86b819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 --2- 172.21.15.57:0/3413672783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86b8103d60 0x7f86b8199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:49.410 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.409+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 >> 172.21.15.57:0/3413672783 conn(0x7f86b80ff4e0 msgr2=0x7f86b8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:49.411 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.410+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 shutdown_connections 2024-09-12T03:05:49.412 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:49.412+0000 7f86c0400640 1 -- 172.21.15.57:0/3413672783 wait complete. 2024-09-12T03:05:49.437 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:05:49.452 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:05:49.452 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T03:05:49.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:49.453 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-09-12T03:05:49.489 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-12T03:05:49.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:49.490 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-12T03:05:49.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:49.593 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2024-09-12T03:05:49.671 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-mon[30361]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:49.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:05:49.672 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setuser ceph since I am not root 2024-09-12T03:05:49.672 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ignoring --setgroup ceph since I am not root 2024-09-12T03:05:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-mon[35320]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:05:49.703 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: ignoring --setuser ceph since I am not root 2024-09-12T03:05:49.703 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: ignoring --setgroup ceph since I am not root 2024-09-12T03:05:49.704 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-mgr[36684]: -- 172.21.15.181:0/677580071 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e1f3b90e0 con 0x557e1f396800 2024-09-12T03:05:49.704 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:49.563+0000 7f694b85b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:05:49.704 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:49.620+0000 7f694b85b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:49.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-mon[35013]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T03:05:49.770 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T03:05:49.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: ignoring --setuser ceph since I am not root 2024-09-12T03:05:49.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: ignoring --setgroup ceph since I am not root 2024-09-12T03:05:49.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-mgr[36821]: -- 172.21.15.204:0/296853414 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fcbeb10e0 con 0x557fcbe8e800 2024-09-12T03:05:49.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:49.564+0000 7f8971b51040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:05:49.771 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:49.621+0000 7f8971b51040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:49.772 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:49.769+0000 7f8971b51040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:05:49.939 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:49.671+0000 7fd93642d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:05:49.939 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:49.744+0000 7fd93642d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:49.942 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T03:05:49.943 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:05:49.943 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:05:49.943 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:05:50.116 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:49.766+0000 7f694b85b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:05:50.116 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:49.832+0000 7f694b85b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:05:50.117 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:49.977+0000 7f694b85b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:50.117 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:50.116+0000 7f694b85b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:05:50.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:49.836+0000 7f8971b51040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:05:50.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:49.985+0000 7f8971b51040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:50.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:50.127+0000 7f8971b51040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:05:50.209 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:49.938+0000 7fd93642d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:05:50.209 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:50.026+0000 7fd93642d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:05:50.336 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:50.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:05:50.477 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-mon[30361]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T03:05:50.477 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:50.208+0000 7fd93642d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:50.477 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:50.392+0000 7fd93642d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:05:50.671 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:05:50.671 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-mon[35320]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T03:05:50.671 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:50.427+0000 7f694b85b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:05:50.671 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:50.488+0000 7f694b85b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:05:50.671 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:50.671+0000 7f694b85b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:05:50.684 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3413672783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T03:05:50.684 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-mon[35013]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-12T03:05:50.685 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:50.437+0000 7f8971b51040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:05:50.685 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:50.498+0000 7f8971b51040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:05:50.685 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:50.684+0000 7f8971b51040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:05:50.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 -- 172.21.15.57:0/2496285085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 msgr2=0x7f3edc101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:50.914 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 --2- 172.21.15.57:0/2496285085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc101e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009900 tx=0x7f3ec802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 -- 172.21.15.57:0/2496285085 shutdown_connections 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 --2- 172.21.15.57:0/2496285085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 0x7f3edc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 --2- 172.21.15.57:0/2496285085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc101e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 --2- 172.21.15.57:0/2496285085 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 0x7f3edc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 -- 172.21.15.57:0/2496285085 >> 172.21.15.57:0/2496285085 conn(0x7f3edc0fd740 msgr2=0x7f3edc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 -- 172.21.15.57:0/2496285085 shutdown_connections 2024-09-12T03:05:50.915 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.913+0000 7f3ee1200640 1 -- 172.21.15.57:0/2496285085 wait complete. 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.914+0000 7f3ee1200640 1 Processor -- start 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.914+0000 7f3ee1200640 1 -- start start 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 0x7f3edc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 0x7f3edc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc111dc0 con 0x7f3edc1079c0 2024-09-12T03:05:50.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc111c40 con 0x7f3edc1019c0 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3ee1200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc111f40 con 0x7f3edc102360 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3edb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3edb400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47212/0 (socket says 172.21.15.57:47212) 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3edaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 0x7f3edc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3edb400640 1 -- 172.21.15.57:0/877137501 learned_addr learned my addr 172.21.15.57:0/877137501 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.915+0000 7f3edbe00640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 0x7f3edc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 msgr2=0x7f3edc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 0x7f3edc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 msgr2=0x7f3edc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 0x7f3edc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 -- 172.21.15.57:0/877137501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc19f540 con 0x7f3edc1019c0 2024-09-12T03:05:50.918 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3edb400640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e9f0 tx=0x7f3ecc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00cd80 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ecc010910 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc19f830 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.916+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018990 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.917+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc1a7110 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.918+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc101e20 con 0x7f3edc1019c0 2024-09-12T03:05:50.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.919+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f3ecc010ac0 con 0x7f3edc1019c0 2024-09-12T03:05:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.922+0000 7f3ed3e00640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 0x7f3ec00a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.922+0000 7f3edaa00640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 msgr2=0x7f3ec00a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:50.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.922+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7f3ecc0cf750 con 0x7f3edc1019c0 2024-09-12T03:05:50.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.922+0000 7f3edaa00640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 0x7f3ec00a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:05:50.925 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:50.924+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ecc09c280 con 0x7f3edc1019c0 2024-09-12T03:05:51.088 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:50.783+0000 7fd93642d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:05:51.088 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:50.860+0000 7fd93642d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:05:51.088 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:51 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:51.087+0000 7fd93642d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:05:51.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.123+0000 7f3edaa00640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 msgr2=0x7f3ec00a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:51.123 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.123+0000 7f3edaa00640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 0x7f3ec00a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:05:51.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.147+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3edc1a72e0 con 0x7f3edc1019c0 2024-09-12T03:05:51.148 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.148+0000 7f3ed3e00640 1 -- 172.21.15.57:0/877137501 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f3ecc09bb80 con 0x7f3edc1019c0 2024-09-12T03:05:51.148 INFO:teuthology.orchestra.run.smithi057.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 msgr2=0x7f3ec00a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 0x7f3ec00a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 msgr2=0x7f3edc19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00e9f0 tx=0x7f3ecc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 shutdown_connections 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7f3ec00a2c50 0x7f3ec00a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f3edc1079c0 0x7f3edc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3edc102360 0x7f3edc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.151 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 --2- 172.21.15.57:0/877137501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3edc1019c0 0x7f3edc19a590 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:51.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 >> 172.21.15.57:0/877137501 conn(0x7f3edc0fd740 msgr2=0x7f3edc10cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:51.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 shutdown_connections 2024-09-12T03:05:51.152 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:51.150+0000 7f3ee1200640 1 -- 172.21.15.57:0/877137501 wait complete. 2024-09-12T03:05:51.672 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:51 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/877137501' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:05:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:51 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/877137501' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:05:51.703 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:51 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:51.550+0000 7f694b85b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:05:51.703 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:51 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:51.616+0000 7f694b85b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:05:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:51 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/877137501' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:05:51.894 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:51.572+0000 7f8971b51040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:05:51.894 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:51.639+0000 7f8971b51040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:05:51.988 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi057.front.sepia.ceph.com 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:05:51.989 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:05:51.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:51.989 DEBUG:teuthology.orchestra.run.smithi057:> ip addr 2024-09-12T03:05:52.026 INFO:teuthology.orchestra.run.smithi057.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:05:52.026 INFO:teuthology.orchestra.run.smithi057.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:05:52.026 INFO:teuthology.orchestra.run.smithi057.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:05:52.026 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 ::1/128 scope host 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:86 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f0 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:05:52.027 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:6b:ff:87 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp7s0f1 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:62 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f0 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: inet 172.21.15.57/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft 42047sec preferred_lft 42047sec 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: inet6 fe80::ec4:7aff:febd:1562/64 scope link 2024-09-12T03:05:52.028 INFO:teuthology.orchestra.run.smithi057.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:05:52.029 INFO:teuthology.orchestra.run.smithi057.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:05:52.029 INFO:teuthology.orchestra.run.smithi057.stdout: link/ether 0c:c4:7a:bd:15:63 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:05:52.029 INFO:teuthology.orchestra.run.smithi057.stdout: altname enp1s0f1 2024-09-12T03:05:52.029 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr flush dev ceph-brx 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set ceph-brx up 2024-09-12T03:05:52.029 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:05:52.030 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:05:52.147 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:52.249 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:52.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:52.259 DEBUG:teuthology.orchestra.run.smithi057:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:05:52.315 INFO:teuthology.orchestra.run.smithi057.stdout:1 2024-09-12T03:05:52.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:52.319 DEBUG:teuthology.orchestra.run.smithi057:> route 2024-09-12T03:05:52.355 INFO:teuthology.orchestra.run.smithi057.stdout:Kernel IP routing table 2024-09-12T03:05:52.355 INFO:teuthology.orchestra.run.smithi057.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:05:52.355 INFO:teuthology.orchestra.run.smithi057.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:05:52.355 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:05:52.355 INFO:teuthology.orchestra.run.smithi057.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:05:52.356 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:05:52.476 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:52.493 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:52.258+0000 7fd93642d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:05:52.493 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:52.345+0000 7fd93642d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:05:52.552 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:52.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:52.561 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-09-12T03:05:52.581 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.322+0000 7f694b85b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:05:52.581 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.437+0000 7f694b85b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:05:52.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:52.597 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list-id 2024-09-12T03:05:52.618 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.357+0000 7f8971b51040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:05:52.618 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.475+0000 7f8971b51040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:05:52.673 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:05:52.674 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:05:52.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:05:52.674 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:05:52.674 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:05:52.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:52.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:52.855 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:05:52.855 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:05:52.855 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:05:52.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:05:52.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:05:52.855 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:05:52.856 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:05:52.856 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:05:52.856 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:05:52.857 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.581+0000 7f694b85b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:52.857 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.641+0000 7f694b85b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:05:52.857 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.789+0000 7f694b85b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:05:52.857 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.857+0000 7f694b85b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:05:52.893 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.618+0000 7f8971b51040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:52.893 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.679+0000 7f8971b51040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:05:52.894 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.829+0000 7f8971b51040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:05:52.934 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:53.068 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:53.077 DEBUG:teuthology.orchestra.run.smithi057:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:05:53.077 DEBUG:teuthology.orchestra.run.smithi057:> set -e 2024-09-12T03:05:53.077 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set brx.0 up 2024-09-12T03:05:53.077 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:05:53.077 DEBUG:teuthology.orchestra.run.smithi057:> ') 2024-09-12T03:05:53.139 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:52.917+0000 7f694b85b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:05:53.139 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.074+0000 7f694b85b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:05:53.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:53.182 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.897+0000 7f8971b51040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:05:53.182 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:52.958+0000 7f8971b51040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:05:53.182 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.116+0000 7f8971b51040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:05:53.182 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.181+0000 7f8971b51040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:05:53.238 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:53.313 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:05:53.313 DEBUG:teuthology.orchestra.run.smithi057:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:05:53.349 INFO:teuthology.orchestra.run.smithi057.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:05:53.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:05:53.349 DEBUG:teuthology.orchestra.run.smithi057:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:05:53.408 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.139+0000 7f694b85b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:05:53.408 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.206+0000 7f694b85b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:05:53.409 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.288+0000 7f694b85b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:53.409 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.347+0000 7f694b85b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:05:53.409 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:53.405+0000 7f694b85b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:05:53.424 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T03:05:53.424 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T03:05:53.425 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7a03ebfb-226f-403a-b746-3dd303bbfeee 2024-09-12T03:05:53.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:53.425 DEBUG:teuthology.orchestra.run.smithi057:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7a03ebfb-226f-403a-b746-3dd303bbfeee /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T03:05:53.449 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.248+0000 7f8971b51040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:05:53.449 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.330+0000 7f8971b51040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:53.449 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.390+0000 7f8971b51040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:05:53.554 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T03:05:53.554 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-12T03:05:53.554 INFO:teuthology.orchestra.run.smithi057.stdout:invalid new device string format 2024-09-12T03:05:53.619 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:53.332+0000 7fd93642d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:05:53.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:53.478+0000 7fd93642d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:05:53.893 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:53.448+0000 7f8971b51040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:05:53.919 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:53.655+0000 7fd93642d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:05:53.919 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:53.733+0000 7fd93642d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:05:54.185 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:53.918+0000 7fd93642d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:05:54.186 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.005+0000 7fd93642d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:05:54.186 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.080+0000 7fd93642d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:invalid new device string format 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: resolved to: "172.21.15.57:6789,172.21.15.181:6789,172.21.15.204:6789" 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.57:6789,172.21.15.181:6789,172.21.15.204:6789:/volumes/qa/sv_1/7a03ebfb-226f-403a-b746-3dd303bbfeee 2024-09-12T03:05:54.201 INFO:teuthology.orchestra.run.smithi057.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=3c25d9c4-70b2-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T03:05:54.203 INFO:teuthology.orchestra.run.smithi057.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:05:54.204 INFO:teuthology.orchestra.run.smithi057.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:05:54.204 INFO:teuthology.orchestra.run.smithi057.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:05:54.204 INFO:teuthology.orchestra.run.smithi057.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:05:54.204 INFO:teuthology.orchestra.run.smithi057.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:05:54.207 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:05:54.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:05:54.207 DEBUG:teuthology.orchestra.run.smithi057:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:05:54.274 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:05:54.357 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:05:54.360 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:54.369 INFO:teuthology.orchestra.run.smithi057.stdout:client15135 2024-09-12T03:05:54.369 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:05:54.370 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:05:54.407 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:05:54.452 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.283+0000 7fd93642d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:05:54.452 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.366+0000 7fd93642d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:05:54.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: Standby manager daemon y restarted 2024-09-12T03:05:54.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: Standby manager daemon y started 2024-09-12T03:05:54.452 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y[36680]: 2024-09-12T03:05:54.188+0000 7f694b85b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: Standby manager daemon y restarted 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: Standby manager daemon y started 2024-09-12T03:05:54.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:05:54.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:54.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:05:54.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:54 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:54.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:05:54.531 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:54.541 INFO:teuthology.orchestra.run.smithi057.stdout:client15135 2024-09-12T03:05:54.542 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:05:54.622 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:05:54.625 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:54.634 INFO:teuthology.orchestra.run.smithi057.stdout:client15135 2024-09-12T03:05:54.634 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968.client15135/status') 2024-09-12T03:05:54.643 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z[36817]: 2024-09-12T03:05:54.239+0000 7f8971b51040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:05:54.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: Standby manager daemon y restarted 2024-09-12T03:05:54.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: Standby manager daemon y started 2024-09-12T03:05:54.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:05:54.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:54.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:05:54.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:54 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.181:0/4055255061' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:54.709 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.451+0000 7fd93642d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:05:54.709 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.556+0000 7fd93642d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:05:54.710 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.633+0000 7fd93642d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:05:54.710 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:54.708+0000 7fd93642d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:05:54.714 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:05:54.736 INFO:teuthology.orchestra.run.smithi057.stdout:instance: client.15135 (3)192.168.0.1:0/3387922647 2024-09-12T03:05:54.736 INFO:teuthology.orchestra.run.smithi057.stdout:blocklisted: false 2024-09-12T03:05:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records in 2024-09-12T03:05:54.736 INFO:teuthology.orchestra.run.smithi057.stderr:0+1 records out 2024-09-12T03:05:54.737 INFO:teuthology.orchestra.run.smithi057.stderr:70 bytes copied, 6.2818e-05 s, 1.1 MB/s 2024-09-12T03:05:54.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:05:54.747 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:05:54.757 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:05:54.757 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:05:54.757 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T03:05:54.916 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 -- 172.21.15.57:0/2417064033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 msgr2=0x7fe4f81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 --2- 172.21.15.57:0/2417064033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f81031d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009860 tx=0x7fe4e402a340 comp rx=0 tx=0).stop 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 -- 172.21.15.57:0/2417064033 shutdown_connections 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 --2- 172.21.15.57:0/2417064033 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 0x7fe4f8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 --2- 172.21.15.57:0/2417064033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f81031d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 --2- 172.21.15.57:0/2417064033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 -- 172.21.15.57:0/2417064033 >> 172.21.15.57:0/2417064033 conn(0x7fe4f80fd2e0 msgr2=0x7fe4f80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 -- 172.21.15.57:0/2417064033 shutdown_connections 2024-09-12T03:05:54.917 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.916+0000 7fe4ff800640 1 -- 172.21.15.57:0/2417064033 wait complete. 2024-09-12T03:05:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 Processor -- start 2024-09-12T03:05:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 -- start start 2024-09-12T03:05:54.919 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 0x7fe4f8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810d7c0 con 0x7fe4f8103710 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810d640 con 0x7fe4f8101b60 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4ff800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810d940 con 0x7fe4f8102d50 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.917+0000 7fe4f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fd800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 0x7fe4f8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47234/0 (socket says 172.21.15.57:47234) 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43626/0 (socket says 172.21.15.57:43626) 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 -- 172.21.15.57:0/3101378166 learned_addr learned my addr 172.21.15.57:0/3101378166 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:54.920 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 msgr2=0x7fe4f80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 msgr2=0x7fe4f8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 0x7fe4f8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 -- 172.21.15.57:0/3101378166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f80753b0 con 0x7fe4f8101b60 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4f7e00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.918+0000 7fe4fce00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9c0 tx=0x7fe4ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.919+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00cd80 con 0x7fe4f8101b60 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.919+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4ec0035f0 con 0x7fe4f8101b60 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.919+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec010690 con 0x7fe4f8101b60 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.919+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f81a1ed0 con 0x7fe4f8101b60 2024-09-12T03:05:54.921 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.919+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81a2280 con 0x7fe4f8101b60 2024-09-12T03:05:54.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.920+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8108ea0 con 0x7fe4f8101b60 2024-09-12T03:05:54.922 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.922+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7fe4ec0037a0 con 0x7fe4f8101b60 2024-09-12T03:05:54.923 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.923+0000 7fe4f5600640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 0x7fe4e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.923+0000 7fe4f7e00640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 msgr2=0x7fe4e00a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.923+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fe4ec0cf790 con 0x7fe4f8101b60 2024-09-12T03:05:54.924 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.923+0000 7fe4f7e00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 0x7fe4e00a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:05:54.928 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:54.927+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4ec09c2c0 con 0x7fe4f8101b60 2024-09-12T03:05:55.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.124+0000 7fe4f7e00640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 msgr2=0x7fe4e00a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:55.124 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.124+0000 7fe4f7e00640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 0x7fe4e00a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:05:55.157 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.154+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fe4f8062ec0 con 0x7fe4f8101b60 2024-09-12T03:05:55.261 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.261+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 21) ==== 137850+0+0 (secure 0 0 0) 0x7fe4ec098230 con 0x7fe4f8101b60 2024-09-12T03:05:55.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.271+0000 7fe4f5600640 1 -- 172.21.15.57:0/3101378166 <== mon.2 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fe4ec0c8b00 con 0x7fe4f8101b60 2024-09-12T03:05:55.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 msgr2=0x7fe4e00a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:05:55.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 0x7fe4e00a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 msgr2=0x7fe4f8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e9c0 tx=0x7fe4ec00ee80 comp rx=0 tx=0).stop 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 shutdown_connections 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fe4e00a2d00 0x7fe4e00a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe4f8103710 0x7fe4f8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4f8102d50 0x7fe4f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 --2- 172.21.15.57:0/3101378166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f8101b60 0x7fe4f8075fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 >> 172.21.15.57:0/3101378166 conn(0x7fe4f80fd2e0 msgr2=0x7fe4f8106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:55.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 shutdown_connections 2024-09-12T03:05:55.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.273+0000 7fe4ff800640 1 -- 172.21.15.57:0/3101378166 wait complete. 2024-09-12T03:05:55.291 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T03:05:55.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 -- 172.21.15.57:0/2844350202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c104f40 msgr2=0x7fbc0c1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:55.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 --2- 172.21.15.57:0/2844350202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c104f40 0x7fbc0c1053c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009860 tx=0x7fbbf802a340 comp rx=0 tx=0).stop 2024-09-12T03:05:55.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 -- 172.21.15.57:0/2844350202 shutdown_connections 2024-09-12T03:05:55.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 --2- 172.21.15.57:0/2844350202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c105900 0x7fbc0c10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.458 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 --2- 172.21.15.57:0/2844350202 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c104f40 0x7fbc0c1053c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 --2- 172.21.15.57:0/2844350202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc0c103d50 0x7fbc0c104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.457+0000 7fbc12200640 1 -- 172.21.15.57:0/2844350202 >> 172.21.15.57:0/2844350202 conn(0x7fbc0c0ff4b0 msgr2=0x7fbc0c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:55.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.458+0000 7fbc12200640 1 -- 172.21.15.57:0/2844350202 shutdown_connections 2024-09-12T03:05:55.459 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.458+0000 7fbc12200640 1 -- 172.21.15.57:0/2844350202 wait complete. 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.458+0000 7fbc12200640 1 Processor -- start 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 -- start start 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c103d50 0x7fbc0c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 0x7fbc0c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc0c105900 0x7fbc0c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:55.460 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10f9b0 con 0x7fbc0c103d50 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10f830 con 0x7fbc0c105900 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc12200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c10fb30 con 0x7fbc0c104f40 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c103d50 0x7fbc0c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 0x7fbc0c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 0x7fbc0c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43654/0 (socket says 172.21.15.57:43654) 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 -- 172.21.15.57:0/2341074105 learned_addr learned my addr 172.21.15.57:0/2341074105 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0be00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc0c105900 0x7fbc0c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc0c105900 msgr2=0x7fbc0c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc0c105900 0x7fbc0c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c103d50 msgr2=0x7fbc0c199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.459+0000 7fbc0aa00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fbc0c103d50 0x7fbc0c199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc0aa00640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c1a1b90 con 0x7fbc0c104f40 2024-09-12T03:05:55.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc0aa00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 0x7fbc0c19a4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8035040 tx=0x7fbbf8033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:55.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8037040 con 0x7fbc0c104f40 2024-09-12T03:05:55.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbf8032770 con 0x7fbc0c104f40 2024-09-12T03:05:55.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1a1e20 con 0x7fbc0c104f40 2024-09-12T03:05:55.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80046c0 con 0x7fbc0c104f40 2024-09-12T03:05:55.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.460+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1a2280 con 0x7fbc0c104f40 2024-09-12T03:05:55.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.462+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c10b0c0 con 0x7fbc0c104f40 2024-09-12T03:05:55.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.463+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) ==== 137850+0+0 (secure 0 0 0) 0x7fbbf8004050 con 0x7fbc0c104f40 2024-09-12T03:05:55.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.465+0000 7fbc03e00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 0x7fbbf00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:55.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.465+0000 7fbc0b400640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 msgr2=0x7fbbf00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:55.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.465+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9208+0+0 (secure 0 0 0) 0x7fbbf80ef3b0 con 0x7fbc0c104f40 2024-09-12T03:05:55.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.465+0000 7fbc0b400640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 0x7fbbf00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:05:55.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.469+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbf80bbee0 con 0x7fbc0c104f40 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: Standby manager daemon z restarted 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: Standby manager daemon z started 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3101378166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-mon[30361]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-09-12T03:05:55.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:05:55.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: Standby manager daemon z restarted 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: Standby manager daemon z started 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3101378166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:55 smithi204 ceph-mon[35013]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-09-12T03:05:55.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.666+0000 7fbc0b400640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 msgr2=0x7fbbf00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.57:6800/480086157 2024-09-12T03:05:55.666 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.666+0000 7fbc0b400640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 0x7fbbf00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:05:55.682 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.682+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbc0c062790 con 0x7fbbf00a2c80 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='mgr.? 172.21.15.204:0/207334265' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: Standby manager daemon z restarted 2024-09-12T03:05:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: Standby manager daemon z started 2024-09-12T03:05:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T03:05:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3101378166' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T03:05:55.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:55 smithi181 ceph-mon[35320]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-09-12T03:05:55.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.720+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 22) ==== 137518+0+0 (secure 0 0 0) 0x7fbbf80b7e50 con 0x7fbc0c104f40 2024-09-12T03:05:55.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.720+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 msgr2=0x7fbbf00a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:05:55.720 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:55.720+0000 7fbc03e00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/480086157,v1:172.21.15.57:6801/480086157] conn(0x7fbbf00a2c80 0x7fbbf00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.002 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:55.699+0000 7fd93642d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:05:56.003 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.001+0000 7fd88d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.001+0000 7fd88d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.001+0000 7fd88d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.001+0000 7fd88d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.001+0000 7fd88d800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.003+0000 7fd891e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.003+0000 7fd891e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.003+0000 7fd891e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.272 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.003+0000 7fd891e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.273 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.003+0000 7fd891e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:56.368 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: Active manager daemon x restarted 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: Activating manager daemon x 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: mgrmap e22: x(active, starting, since 0.0129064s), standbys: y, z 2024-09-12T03:05:56.369 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:56.370 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:56.371 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:56.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:56.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:56.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:05:56.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:05:56.372 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:05:56.373 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:05:56.374 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: Manager daemon x is now available 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:56.375 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.376 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.376 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.376 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:56 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.377 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:05:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: Active manager daemon x restarted 2024-09-12T03:05:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: Activating manager daemon x 2024-09-12T03:05:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:05:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: mgrmap e22: x(active, starting, since 0.0129064s), standbys: y, z 2024-09-12T03:05:56.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:05:56.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:05:56.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:05:56.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:56.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:56.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:56.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:56.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:56.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:56.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:56.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:05:56.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:05:56.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:05:56.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: Manager daemon x is now available 2024-09-12T03:05:56.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:56.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:56 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T03:05:56.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:05:56.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: Active manager daemon x restarted 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: Activating manager daemon x 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: mgrmap e22: x(active, starting, since 0.0129064s), standbys: y, z 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:05:56.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:05:56.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:05:56.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: Manager daemon x is now available 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:05:56.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.724+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mon.1 v2:172.21.15.204:3300/0 8 ==== mgrmap(e 23) ==== 137645+0+0 (secure 0 0 0) 0x7fbbf80ecb00 con 0x7fbc0c104f40 2024-09-12T03:05:56.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.724+0000 7fbc03e00640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fbbf0051920 0x7fbbf0051d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:56.726 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.724+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fbbf80a61d0 con 0x7fbbf0051920 2024-09-12T03:05:56.727 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.727+0000 7fbc0b400640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fbbf0051920 0x7fbbf0051d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:56.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.733+0000 7fbc0b400640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fbbf0051920 0x7fbbf0051d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c104da0 tx=0x7fbbfc009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:56.757 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.757+0000 7fbc03e00640 1 -- 172.21.15.57:0/2341074105 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fbbf80a61d0 con 0x7fbbf0051920 2024-09-12T03:05:56.757 INFO:teuthology.orchestra.run.smithi057.stdout:{} 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fbbf0051920 msgr2=0x7fbbf0051d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fbbf0051920 0x7fbbf0051d00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c104da0 tx=0x7fbbfc009260 comp rx=0 tx=0).stop 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 msgr2=0x7fbc0c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 --2- 172.21.15.57:0/2341074105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc0c104f40 0x7fbc0c19a4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8035040 tx=0x7fbbf8033d90 comp rx=0 tx=0).stop 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc0be00640 1 -- 172.21.15.57:0/2341074105 reap_dead start 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 shutdown_connections 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 >> 172.21.15.57:0/2341074105 conn(0x7fbc0c0ff4b0 msgr2=0x7fbc0c108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 shutdown_connections 2024-09-12T03:05:56.760 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.759+0000 7fbc12200640 1 -- 172.21.15.57:0/2341074105 wait complete. 2024-09-12T03:05:56.775 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:05:56.786 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:05:56.831 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:05:56.831 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:05:56.831 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T03:05:56.996 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.995+0000 7f5d90400640 1 -- 172.21.15.57:0/3637025463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 msgr2=0x7f5d88104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 --2- 172.21.15.57:0/3637025463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88104160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009900 tx=0x7f5d7c02a410 comp rx=0 tx=0).stop 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 -- 172.21.15.57:0/3637025463 shutdown_connections 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 --2- 172.21.15.57:0/3637025463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d8810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 --2- 172.21.15.57:0/3637025463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 0x7f5d881053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 --2- 172.21.15.57:0/3637025463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88104160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 -- 172.21.15.57:0/3637025463 >> 172.21.15.57:0/3637025463 conn(0x7f5d880ff4e0 msgr2=0x7f5d88101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 -- 172.21.15.57:0/3637025463 shutdown_connections 2024-09-12T03:05:56.997 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.996+0000 7f5d90400640 1 -- 172.21.15.57:0/3637025463 wait complete. 2024-09-12T03:05:56.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 Processor -- start 2024-09-12T03:05:56.999 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 -- start start 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 0x7f5d8819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d881a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810fa10 con 0x7f5d88103d60 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810f890 con 0x7f5d88104f50 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.997+0000 7f5d90400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810fb90 con 0x7f5d88105910 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8da00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8da00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55420/0 (socket says 172.21.15.57:55420) 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d881a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8da00640 1 -- 172.21.15.57:0/3199742817 learned_addr learned my addr 172.21.15.57:0/3199742817 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:57.000 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8d000640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 0x7f5d8819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 -- 172.21.15.57:0/3199742817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 msgr2=0x7f5d8819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 0x7f5d8819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 -- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 msgr2=0x7f5d88199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 -- 172.21.15.57:0/3199742817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d881a1b90 con 0x7f5d88105910 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.998+0000 7f5d8e400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d881a1520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d8000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.999+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000ce20 con 0x7f5d88105910 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.999+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d80005020 con 0x7f5d88105910 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.999+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80010720 con 0x7f5d88105910 2024-09-12T03:05:57.001 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.999+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d881a1e80 con 0x7f5d88105910 2024-09-12T03:05:57.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:56.999+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d881a22f0 con 0x7f5d88105910 2024-09-12T03:05:57.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.000+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8810b080 con 0x7f5d88105910 2024-09-12T03:05:57.002 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.002+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) ==== 137645+0+0 (secure 0 0 0) 0x7f5d80005540 con 0x7f5d88105910 2024-09-12T03:05:57.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.004+0000 7f5d86000640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 0x7f5d700a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.004+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5d800cf7d0 con 0x7f5d88105910 2024-09-12T03:05:57.004 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.004+0000 7f5d8da00640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 0x7f5d700a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.008 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.005+0000 7f5d8da00640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 0x7f5d700a4f80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000c00 tx=0x7f5d7c0315b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:57.009 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.007+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d8009c510 con 0x7f5d88105910 2024-09-12T03:05:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.742+0000 7fd88ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.742+0000 7fd88ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.742+0000 7fd88ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.742+0000 7fd88ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:57.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:56.742+0000 7fd88ba00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:05:57.194 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.194+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f5d8802d7c0 con 0x7f5d88105910 2024-09-12T03:05:57.200 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.200+0000 7f5d86000640 1 -- 172.21.15.57:0/3199742817 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f5d8009be10 con 0x7f5d88105910 2024-09-12T03:05:57.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 msgr2=0x7f5d700a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.203 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 0x7f5d700a4f80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000c00 tx=0x7f5d7c0315b0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 msgr2=0x7f5d881a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d881a1520 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009fd0 tx=0x7f5d8000ef10 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 shutdown_connections 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d88105910 0x7f5d881a1520 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d88104f50 0x7f5d8819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5d700a2ac0 0x7f5d700a4f80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 --2- 172.21.15.57:0/3199742817 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5d88103d60 0x7f5d88199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 >> 172.21.15.57:0/3199742817 conn(0x7f5d880ff4e0 msgr2=0x7f5d88108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 shutdown_connections 2024-09-12T03:05:57.204 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.203+0000 7f5d90400640 1 -- 172.21.15.57:0/3199742817 wait complete. 2024-09-12T03:05:57.227 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 -- 172.21.15.57:0/1577732617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc4107830 msgr2=0x7f4fc4109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/1577732617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc4107830 0x7f4fc4109ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009ca0 tx=0x7f4fbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 -- 172.21.15.57:0/1577732617 shutdown_connections 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/1577732617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc4107830 0x7f4fc4109ca0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/1577732617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4104f00 0x7f4fc41072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/1577732617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc41025d0 0x7f4fc41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.394 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 -- 172.21.15.57:0/1577732617 >> 172.21.15.57:0/1577732617 conn(0x7f4fc40fc1b0 msgr2=0x7f4fc40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:57.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 -- 172.21.15.57:0/1577732617 shutdown_connections 2024-09-12T03:05:57.395 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.393+0000 7f4fc8c00640 1 -- 172.21.15.57:0/1577732617 wait complete. 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.394+0000 7f4fc8c00640 1 Processor -- start 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.394+0000 7f4fc8c00640 1 -- start start 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.394+0000 7f4fc8c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.394+0000 7f4fc8c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 0x7f4fc4100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4107830 0x7f4fc4100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc8c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410d860 con 0x7f4fc41025d0 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc8c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410d6e0 con 0x7f4fc4104f00 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410d9e0 con 0x7f4fc4107830 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.397 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55438/0 (socket says 172.21.15.57:55438) 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 -- 172.21.15.57:0/2398010604 learned_addr learned my addr 172.21.15.57:0/2398010604 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc2800640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4107830 0x7f4fc4100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1400640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 0x7f4fc4100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 -- 172.21.15.57:0/2398010604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4107830 msgr2=0x7f4fc4100ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4107830 0x7f4fc4100ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 -- 172.21.15.57:0/2398010604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 msgr2=0x7f4fc4100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 0x7f4fc4100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1e00640 1 -- 172.21.15.57:0/2398010604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc4101350 con 0x7f4fc41025d0 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.395+0000 7f4fc1400640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 0x7f4fc4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fc1e00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400ea60 tx=0x7f4fb400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:57.398 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400cde0 con 0x7f4fc41025d0 2024-09-12T03:05:57.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb40035f0 con 0x7f4fc41025d0 2024-09-12T03:05:57.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a1f60 con 0x7f4fc41025d0 2024-09-12T03:05:57.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4010670 con 0x7f4fc41025d0 2024-09-12T03:05:57.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.396+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a23c0 con 0x7f4fc41025d0 2024-09-12T03:05:57.399 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.397+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc40fdca0 con 0x7f4fc41025d0 2024-09-12T03:05:57.401 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.401+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 23) ==== 137645+0+0 (secure 0 0 0) 0x7f4fb40037a0 con 0x7f4fc41025d0 2024-09-12T03:05:57.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.402+0000 7f4fbaa00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 0x7f4fa80a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.402+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f4fb40cf360 con 0x7f4fc41025d0 2024-09-12T03:05:57.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.402+0000 7f4fc1400640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 0x7f4fa80a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.403 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.403+0000 7f4fc1400640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 0x7f4fa80a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4101cb0 tx=0x7f4fb0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:57.407 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.406+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fb409c0a0 con 0x7f4fc41025d0 2024-09-12T03:05:57.632 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.631+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f4fc4062630 con 0x7f4fc41025d0 2024-09-12T03:05:57.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.640+0000 7f4fbaa00640 1 -- 172.21.15.57:0/2398010604 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f4fb409b9a0 con 0x7f4fc41025d0 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 msgr2=0x7f4fa80a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 0x7f4fa80a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4101cb0 tx=0x7f4fb0005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 msgr2=0x7f4fc4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400ea60 tx=0x7f4fb400ef20 comp rx=0 tx=0).stop 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 shutdown_connections 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fc4107830 0x7f4fc4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc4104f00 0x7f4fc4100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4fa80a2ac0 0x7f4fa80a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 --2- 172.21.15.57:0/2398010604 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4fc41025d0 0x7f4fc4101f40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 >> 172.21.15.57:0/2398010604 conn(0x7f4fc40fc1b0 msgr2=0x7f4fc4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.642+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 shutdown_connections 2024-09-12T03:05:57.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.643+0000 7f4fc8c00640 1 -- 172.21.15.57:0/2398010604 wait complete. 2024-09-12T03:05:57.660 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T03:05:57.771 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: mgrmap e23: x(active, since 1.01743s), standbys: y, z 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: [12/Sep/2024:03:05:57] ENGINE Bus STARTING 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: [12/Sep/2024:03:05:57] ENGINE Serving on https://172.21.15.57:7150 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: [12/Sep/2024:03:05:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: Traceback (most recent call last): 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._run(expiration_interval) 2024-09-12T03:05:57.772 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: s = self.context.wrap_socket( 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: return self.sslsocket_class._create( 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: self.do_handshake() 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: self._sslobj.do_handshake() 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:05:57.773 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2398010604' entity='client.admin' 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:57.499+0000 7fd871600640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:05:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Traceback (most recent call last): 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._connections.run(self.expiration_interval) 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._run(expiration_interval) 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:05:57.774 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: s = self.context.wrap_socket( 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: return self.sslsocket_class._create( 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.do_handshake() 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self._sslobj.do_handshake() 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:05:57.775 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:05:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:05:57.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 -- 172.21.15.57:0/2071954026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870104f50 msgr2=0x7f18701053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 --2- 172.21.15.57:0/2071954026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870104f50 0x7f18701053d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1858009900 tx=0x7f185802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 -- 172.21.15.57:0/2071954026 shutdown_connections 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 --2- 172.21.15.57:0/2071954026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870105910 0x7f187010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 --2- 172.21.15.57:0/2071954026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870104f50 0x7f18701053d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 --2- 172.21.15.57:0/2071954026 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870103d60 0x7f1870104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 -- 172.21.15.57:0/2071954026 >> 172.21.15.57:0/2071954026 conn(0x7f18700ff4e0 msgr2=0x7f1870101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 -- 172.21.15.57:0/2071954026 shutdown_connections 2024-09-12T03:05:57.829 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.824+0000 7f1875e00640 1 -- 172.21.15.57:0/2071954026 wait complete. 2024-09-12T03:05:57.831 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.825+0000 7f1875e00640 1 Processor -- start 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.825+0000 7f1875e00640 1 -- start start 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870104f50 0x7f187019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 0x7f18701a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f187010fa30 con 0x7f1870105910 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f187010f8b0 con 0x7f1870103d60 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1875e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f187010fbb0 con 0x7f1870104f50 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f1867e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870104f50 0x7f187019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47436/0 (socket says 172.21.15.57:47436) 2024-09-12T03:05:57.832 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 -- 172.21.15.57:0/2313629542 learned_addr learned my addr 172.21.15.57:0/2313629542 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186f400640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 0x7f18701a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 -- 172.21.15.57:0/2313629542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870104f50 msgr2=0x7f187019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870104f50 0x7f187019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 -- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 msgr2=0x7f18701a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 0x7f18701a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186ea00640 1 -- 172.21.15.57:0/2313629542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18701a1c10 con 0x7f1870103d60 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.826+0000 7f186f400640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 0x7f18701a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:57.833 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f186ea00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f186000e9c0 tx=0x7f186000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186000cd80 con 0x7f1870103d60 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18600035f0 con 0x7f1870103d60 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18701a1f00 con 0x7f1870103d60 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860010690 con 0x7f1870103d60 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.827+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18701a2370 con 0x7f1870103d60 2024-09-12T03:05:57.834 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.828+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187010b080 con 0x7f1870103d60 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.832+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 23) ==== 137645+0+0 (secure 0 0 0) 0x7f18600037a0 con 0x7f1870103d60 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.833+0000 7f1874a00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 0x7f18540a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.833+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f18600cf520 con 0x7f1870103d60 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.834+0000 7f1867e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 0x7f18540a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.834+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f186009c260 con 0x7f1870103d60 2024-09-12T03:05:57.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:57.835+0000 7f1867e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 0x7f18540a4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1858031440 tx=0x7f18580315b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:58.037 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.037+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f187002d7c0 con 0x7f1870103d60 2024-09-12T03:05:58.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.047+0000 7f1874a00640 1 -- 172.21.15.57:0/2313629542 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f186009bb60 con 0x7f1870103d60 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 msgr2=0x7f18540a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 0x7f18540a4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1858031440 tx=0x7f18580315b0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 msgr2=0x7f1870199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f186000e9c0 tx=0x7f186000ee80 comp rx=0 tx=0).stop 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 shutdown_connections 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f18540a2ac0 0x7f18540a4f80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1870105910 0x7f18701a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1870104f50 0x7f187019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 --2- 172.21.15.57:0/2313629542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1870103d60 0x7f1870199ff0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.049+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 >> 172.21.15.57:0/2313629542 conn(0x7f18700ff4e0 msgr2=0x7f1870108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.050+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 shutdown_connections 2024-09-12T03:05:58.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.050+0000 7f1875e00640 1 -- 172.21.15.57:0/2313629542 wait complete. 2024-09-12T03:05:58.067 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T03:05:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: mgrmap e23: x(active, since 1.01743s), standbys: y, z 2024-09-12T03:05:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: [12/Sep/2024:03:05:57] ENGINE Bus STARTING 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: [12/Sep/2024:03:05:57] ENGINE Serving on https://172.21.15.57:7150 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: [12/Sep/2024:03:05:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: Traceback (most recent call last): 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._run(expiration_interval) 2024-09-12T03:05:58.144 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: s = self.context.wrap_socket( 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:05:58.145 INFO:journalctl@ceph.mon.c.smithi204.stdout: return self.sslsocket_class._create( 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout: self.do_handshake() 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._sslobj.do_handshake() 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:05:58.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:57 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2398010604' entity='client.admin' 2024-09-12T03:05:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: mgrmap e23: x(active, since 1.01743s), standbys: y, z 2024-09-12T03:05:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' 2024-09-12T03:05:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: [12/Sep/2024:03:05:57] ENGINE Bus STARTING 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: [12/Sep/2024:03:05:57] ENGINE Serving on https://172.21.15.57:7150 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: [12/Sep/2024:03:05:57] ENGINE Error in HTTPServer.serve 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: Traceback (most recent call last): 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._run(expiration_interval) 2024-09-12T03:05:58.204 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: s = self.context.wrap_socket( 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: return self.sslsocket_class._create( 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: self.do_handshake() 2024-09-12T03:05:58.205 INFO:journalctl@ceph.mon.b.smithi181.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:05:58.206 INFO:journalctl@ceph.mon.b.smithi181.stdout: self._sslobj.do_handshake() 2024-09-12T03:05:58.206 INFO:journalctl@ceph.mon.b.smithi181.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:05:58.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:57 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2398010604' entity='client.admin' 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 -- 172.21.15.57:0/4264081485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 msgr2=0x7ffa3c078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 --2- 172.21.15.57:0/4264081485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c078250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009900 tx=0x7ffa2802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 -- 172.21.15.57:0/4264081485 shutdown_connections 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 --2- 172.21.15.57:0/4264081485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c078790 0x7ffa3c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 --2- 172.21.15.57:0/4264081485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c078250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 --2- 172.21.15.57:0/4264081485 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 -- 172.21.15.57:0/4264081485 >> 172.21.15.57:0/4264081485 conn(0x7ffa3c0ff4e0 msgr2=0x7ffa3c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:58.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 -- 172.21.15.57:0/4264081485 shutdown_connections 2024-09-12T03:05:58.228 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.226+0000 7ffa42a00640 1 -- 172.21.15.57:0/4264081485 wait complete. 2024-09-12T03:05:58.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 Processor -- start 2024-09-12T03:05:58.229 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 -- start start 2024-09-12T03:05:58.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 0x7ffa3c1a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c113e10 con 0x7ffa3c078790 2024-09-12T03:05:58.230 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c113c90 con 0x7ffa3c076bb0 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.227+0000 7ffa42a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c113f90 con 0x7ffa3c077df0 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:37938/0 (socket says 172.21.15.57:37938) 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54850/0 (socket says 172.21.15.57:54850) 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3b400640 1 -- 172.21.15.57:0/2779827756 learned_addr learned my addr 172.21.15.57:0/2779827756 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa40a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 0x7ffa3c1a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 -- 172.21.15.57:0/2779827756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 msgr2=0x7ffa3c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 -- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 msgr2=0x7ffa3c1a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 0x7ffa3c1a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.231 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 -- 172.21.15.57:0/2779827756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c1a6050 con 0x7ffa3c076bb0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa40a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 0x7ffa3c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.228+0000 7ffa3be00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e9f0 tx=0x7ffa2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.229+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00ccc0 con 0x7ffa3c076bb0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.229+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c1a6340 con 0x7ffa3c076bb0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.229+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2c010910 con 0x7ffa3c076bb0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.229+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c018960 con 0x7ffa3c076bb0 2024-09-12T03:05:58.232 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.229+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a6890 con 0x7ffa3c076bb0 2024-09-12T03:05:58.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.230+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c1106d0 con 0x7ffa3c076bb0 2024-09-12T03:05:58.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.232+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 23) ==== 137645+0+0 (secure 0 0 0) 0x7ffa2c010ac0 con 0x7ffa3c076bb0 2024-09-12T03:05:58.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.233+0000 7ffa38c00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 0x7ffa200a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.233 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.233+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ffa2c0cf550 con 0x7ffa3c076bb0 2024-09-12T03:05:58.234 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.234+0000 7ffa3b400640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 0x7ffa200a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.235 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.235+0000 7ffa3b400640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 0x7ffa200a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c19f940 tx=0x7ffa280315b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:58.237 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.237+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa2c09c2d0 con 0x7ffa3c076bb0 2024-09-12T03:05:58.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.503+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffa3c062630 con 0x7ffa200a2a40 2024-09-12T03:05:58.577 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.576+0000 7ffa38c00640 1 -- 172.21.15.57:0/2779827756 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffa3c062630 con 0x7ffa200a2a40 2024-09-12T03:05:58.577 INFO:teuthology.orchestra.run.smithi057.stdout:Schedule set for path / 2024-09-12T03:05:58.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 msgr2=0x7ffa200a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 0x7ffa200a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c19f940 tx=0x7ffa280315b0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.580 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 msgr2=0x7ffa3c19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00e9f0 tx=0x7ffa2c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 shutdown_connections 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa200a2a40 0x7ffa200a4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa3c078790 0x7ffa3c1a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa3c077df0 0x7ffa3c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 --2- 172.21.15.57:0/2779827756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa3c076bb0 0x7ffa3c19e430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 >> 172.21.15.57:0/2779827756 conn(0x7ffa3c0ff4e0 msgr2=0x7ffa3c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 shutdown_connections 2024-09-12T03:05:58.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.580+0000 7ffa42a00640 1 -- 172.21.15.57:0/2779827756 wait complete. 2024-09-12T03:05:58.598 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T03:05:58.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 -- 172.21.15.57:0/3416662269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e381023c0 msgr2=0x7f8e381047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3416662269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e381023c0 0x7f8e381047b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009860 tx=0x7f8e2802a340 comp rx=0 tx=0).stop 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 -- 172.21.15.57:0/3416662269 shutdown_connections 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3416662269 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38107620 0x7f8e38109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3416662269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e38104cf0 0x7f8e381070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3416662269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e381023c0 0x7f8e381047b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.776+0000 7f8e3d400640 1 -- 172.21.15.57:0/3416662269 >> 172.21.15.57:0/3416662269 conn(0x7f8e380fbfa0 msgr2=0x7f8e380fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.777+0000 7f8e3d400640 1 -- 172.21.15.57:0/3416662269 shutdown_connections 2024-09-12T03:05:58.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.777+0000 7f8e3d400640 1 -- 172.21.15.57:0/3416662269 wait complete. 2024-09-12T03:05:58.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.777+0000 7f8e3d400640 1 Processor -- start 2024-09-12T03:05:58.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 -- start start 2024-09-12T03:05:58.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e381023c0 0x7f8e38101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 0x7f8e38100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 0x7f8e381008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d600 con 0x7f8e38104cf0 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d480 con 0x7f8e381023c0 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3810d780 con 0x7f8e38107620 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e37400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 0x7f8e38100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e37400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 0x7f8e38100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51500/0 (socket says 172.21.15.57:51500) 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e37400640 1 -- 172.21.15.57:0/3386446423 learned_addr learned my addr 172.21.15.57:0/3386446423 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e3ca00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 0x7f8e381008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.778+0000 7f8e37e00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e381023c0 0x7f8e38101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 -- 172.21.15.57:0/3386446423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e381023c0 msgr2=0x7f8e38101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e381023c0 0x7f8e38101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 -- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 msgr2=0x7f8e38100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:58.781 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 0x7f8e38100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 -- 172.21.15.57:0/3386446423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e38100df0 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3ca00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 0x7f8e381008b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005fd0 tx=0x7f8e3000c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004b20 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30003e50 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381010e0 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000b570 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.779+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381a67c0 con 0x7f8e38107620 2024-09-12T03:05:58.782 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.780+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e380fda90 con 0x7f8e38107620 2024-09-12T03:05:58.783 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.783+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138004+0+0 (secure 0 0 0) 0x7f8e3001d020 con 0x7f8e38107620 2024-09-12T03:05:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.784+0000 7f8e34c00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 0x7f8e1c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:58.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.784+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f8e300cf820 con 0x7f8e38107620 2024-09-12T03:05:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.784+0000 7f8e37e00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 0x7f8e1c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:58.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.785+0000 7f8e37e00640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 0x7f8e1c0a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009830 tx=0x7f8e28030550 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:58.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:58.788+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e3009bec0 con 0x7f8e38107620 2024-09-12T03:05:59.015 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.014+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8e38062790 con 0x7f8e1c0a2e00 2024-09-12T03:05:59.091 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.091+0000 7f8e34c00640 1 -- 172.21.15.57:0/3386446423 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8e38062790 con 0x7f8e1c0a2e00 2024-09-12T03:05:59.091 INFO:teuthology.orchestra.run.smithi057.stdout:Retention added to path / 2024-09-12T03:05:59.093 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 msgr2=0x7f8e1c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 0x7f8e1c0a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009830 tx=0x7f8e28030550 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 msgr2=0x7f8e381008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 0x7f8e381008b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005fd0 tx=0x7f8e3000c740 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 shutdown_connections 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e38107620 0x7f8e381008b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8e1c0a2e00 0x7f8e1c0a52c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e38104cf0 0x7f8e38100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 --2- 172.21.15.57:0/3386446423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e381023c0 0x7f8e38101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 >> 172.21.15.57:0/3386446423 conn(0x7f8e380fbfa0 msgr2=0x7f8e38102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 shutdown_connections 2024-09-12T03:05:59.094 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.093+0000 7f8e3d400640 1 -- 172.21.15.57:0/3386446423 wait complete. 2024-09-12T03:05:59.110 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:05:59.158 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: [12/Sep/2024:03:05:57] ENGINE Serving on http://172.21.15.57:8765 2024-09-12T03:05:59.158 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: [12/Sep/2024:03:05:57] ENGINE Bus STARTED 2024-09-12T03:05:59.158 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:59.158 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.158 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.159 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.160 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:05:58 smithi057 ceph-mon[30361]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-09-12T03:05:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: [12/Sep/2024:03:05:57] ENGINE Serving on http://172.21.15.57:8765 2024-09-12T03:05:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: [12/Sep/2024:03:05:57] ENGINE Bus STARTED 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.206 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:05:58 smithi181 ceph-mon[35320]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1157167670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4104cf0 msgr2=0x7fc6e41070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1157167670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4104cf0 0x7fc6e41070e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009900 tx=0x7fc6d002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1157167670 shutdown_connections 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1157167670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e4107620 0x7fc6e4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1157167670 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4104cf0 0x7fc6e41070e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1157167670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e41023c0 0x7fc6e41047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1157167670 >> 172.21.15.57:0/1157167670 conn(0x7fc6e40fbfa0 msgr2=0x7fc6e40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1157167670 shutdown_connections 2024-09-12T03:05:59.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.266+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1157167670 wait complete. 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.267+0000 7fc6e3e00640 1 Processor -- start 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.267+0000 7fc6e3e00640 1 -- start start 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 0x7fc6e4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 0x7fc6e4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 0x7fc6e419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d620 con 0x7fc6e4107620 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d4a0 con 0x7fc6e41023c0 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410d7a0 con 0x7fc6e4104cf0 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 0x7fc6e4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 0x7fc6e419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.269 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 0x7fc6e4197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54892/0 (socket says 172.21.15.57:54892) 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2a00640 1 -- 172.21.15.57:0/1840844293 learned_addr learned my addr 172.21.15.57:0/1840844293 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 0x7fc6e4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 -- 172.21.15.57:0/1840844293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 msgr2=0x7fc6e4197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 0x7fc6e4197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 -- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 msgr2=0x7fc6e419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 0x7fc6e419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e2000640 1 -- 172.21.15.57:0/1840844293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e419f930 con 0x7fc6e4104cf0 2024-09-12T03:05:59.270 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.268+0000 7fc6e3400640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 0x7fc6e419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e2000640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 0x7fc6e4198260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009230 tx=0x7fc6d0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d003f030 con 0x7fc6e4104cf0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d0003ce0 con 0x7fc6e4104cf0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e419fbc0 con 0x7fc6e4104cf0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0035070 con 0x7fc6e4104cf0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.269+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e41a0080 con 0x7fc6e4104cf0 2024-09-12T03:05:59.271 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.270+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e40fda90 con 0x7fc6e4104cf0 2024-09-12T03:05:59.272 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.272+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) ==== 138004+0+0 (secure 0 0 0) 0x7fc6d0042030 con 0x7fc6e4104cf0 2024-09-12T03:05:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.273+0000 7fc6e8a00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 0x7fc6c80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.274+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fc6d00eff10 con 0x7fc6e4104cf0 2024-09-12T03:05:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.274+0000 7fc6e2a00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 0x7fc6c80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.274+0000 7fc6e2a00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 0x7fc6c80a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e41991f0 tx=0x7fc6d4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:59.276 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.275+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6d00bbf60 con 0x7fc6e4104cf0 2024-09-12T03:05:59.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: [12/Sep/2024:03:05:57] ENGINE Serving on http://172.21.15.57:8765 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: [12/Sep/2024:03:05:57] ENGINE Bus STARTED 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='client.25259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:05:59.396 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:05:58 smithi204 ceph-mon[35013]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-09-12T03:05:59.506 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.506+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc6e4062630 con 0x7fc6c80a2d80 2024-09-12T03:05:59.514 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.513+0000 7fc6e8a00640 1 -- 172.21.15.57:0/1840844293 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fc6e4062630 con 0x7fc6c80a2d80 2024-09-12T03:05:59.514 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:05:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 msgr2=0x7fc6c80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 0x7fc6c80a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e41991f0 tx=0x7fc6d4005f90 comp rx=0 tx=0).stop 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 msgr2=0x7fc6e4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 0x7fc6e4198260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009230 tx=0x7fc6d0032ce0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 shutdown_connections 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc6c80a2d80 0x7fc6c80a5240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc6e4107620 0x7fc6e419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.516 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6e4104cf0 0x7fc6e4198260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 --2- 172.21.15.57:0/1840844293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6e41023c0 0x7fc6e4197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.515+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 >> 172.21.15.57:0/1840844293 conn(0x7fc6e40fbfa0 msgr2=0x7fc6e4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:59.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.516+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 shutdown_connections 2024-09-12T03:05:59.517 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.516+0000 7fc6e3e00640 1 -- 172.21.15.57:0/1840844293 wait complete. 2024-09-12T03:05:59.534 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:05:59.693 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 -- 172.21.15.57:0/2954321495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80103d60 msgr2=0x7f5f80104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 --2- 172.21.15.57:0/2954321495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80103d60 0x7f5f80104160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009900 tx=0x7f5f7002a410 comp rx=0 tx=0).stop 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 -- 172.21.15.57:0/2954321495 shutdown_connections 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 --2- 172.21.15.57:0/2954321495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80105910 0x7f5f8010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 --2- 172.21.15.57:0/2954321495 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80104f50 0x7f5f801053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 --2- 172.21.15.57:0/2954321495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80103d60 0x7f5f80104160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 -- 172.21.15.57:0/2954321495 >> 172.21.15.57:0/2954321495 conn(0x7f5f800ff4e0 msgr2=0x7f5f80101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 -- 172.21.15.57:0/2954321495 shutdown_connections 2024-09-12T03:05:59.694 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.693+0000 7f5f85000640 1 -- 172.21.15.57:0/2954321495 wait complete. 2024-09-12T03:05:59.695 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 Processor -- start 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 -- start start 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 0x7f5f80199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 0x7f5f8019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 0x7f5f801a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010fa30 con 0x7f5f80103d60 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010f8b0 con 0x7f5f80105910 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.694+0000 7f5f85000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8010fbb0 con 0x7f5f80104f50 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 0x7f5f80199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 0x7f5f801a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 0x7f5f8019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 0x7f5f80199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51536/0 (socket says 172.21.15.57:51536) 2024-09-12T03:05:59.696 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7de00640 1 -- 172.21.15.57:0/1210225824 learned_addr learned my addr 172.21.15.57:0/1210225824 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 -- 172.21.15.57:0/1210225824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 msgr2=0x7f5f8019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 0x7f5f8019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 -- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 msgr2=0x7f5f80199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 0x7f5f80199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 -- 172.21.15.57:0/1210225824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f801a1bc0 con 0x7f5f80105910 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7d400640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 0x7f5f8019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:05:59.697 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.695+0000 7f5f7e800640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 0x7f5f801a1550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fd0 tx=0x7f5f7800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.696+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800ce20 con 0x7f5f80105910 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.696+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f78005020 con 0x7f5f80105910 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.696+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f801a1eb0 con 0x7f5f80105910 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.696+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78010720 con 0x7f5f80105910 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.696+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f801a23c0 con 0x7f5f80105910 2024-09-12T03:05:59.698 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.697+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8010b080 con 0x7f5f80105910 2024-09-12T03:05:59.699 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.699+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 24) ==== 138004+0+0 (secure 0 0 0) 0x7f5f78005540 con 0x7f5f80105910 2024-09-12T03:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.701+0000 7f5f76a00640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 0x7f5f640a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.701+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5f780cff40 con 0x7f5f80105910 2024-09-12T03:05:59.701 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.701+0000 7f5f7de00640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 0x7f5f640a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:59.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.702+0000 7f5f7de00640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 0x7f5f640a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f70000c00 tx=0x7f5f700315b0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:59.702 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.702+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f7809c5e0 con 0x7f5f80105910 2024-09-12T03:05:59.948 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.948+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5f80062630 con 0x7f5f640a2ec0 2024-09-12T03:05:59.957 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.956+0000 7f5f76a00640 1 -- 172.21.15.57:0/1210225824 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5f80062630 con 0x7f5f640a2ec0 2024-09-12T03:05:59.957 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.958+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 msgr2=0x7f5f640a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.958+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 0x7f5f640a5380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f70000c00 tx=0x7f5f700315b0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.958+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 msgr2=0x7f5f801a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.958+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 0x7f5f801a1550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fd0 tx=0x7f5f7800ef10 comp rx=0 tx=0).stop 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 shutdown_connections 2024-09-12T03:05:59.959 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f80105910 0x7f5f801a1550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f80104f50 0x7f5f8019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5f640a2ec0 0x7f5f640a5380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 --2- 172.21.15.57:0/1210225824 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5f80103d60 0x7f5f80199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 >> 172.21.15.57:0/1210225824 conn(0x7f5f800ff4e0 msgr2=0x7f5f80108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 shutdown_connections 2024-09-12T03:05:59.960 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:05:59.959+0000 7f5f85000640 1 -- 172.21.15.57:0/1210225824 wait complete. 2024-09-12T03:05:59.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T03:06:00.034 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T03:06:00.046 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T03:06:00.103 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:06:00.103 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T03:06:00.107 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:06:00.155 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:06:00.155 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:06:00.155 INFO:tasks.workunit:timeout=3h 2024-09-12T03:06:00.155 INFO:tasks.workunit:cleanup=True 2024-09-12T03:06:00.156 DEBUG:teuthology.orchestra.run.smithi057:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T03:06:00.200 INFO:teuthology.orchestra.run.smithi057.stdout:Access: 2024-09-12 03:04:08.312586243 +0000 2024-09-12T03:06:00.201 INFO:teuthology.orchestra.run.smithi057.stdout:Modify: 2024-09-12 03:04:08.312586243 +0000 2024-09-12T03:06:00.201 INFO:teuthology.orchestra.run.smithi057.stdout:Change: 2024-09-12 03:05:54.267386601 +0000 2024-09-12T03:06:00.201 INFO:teuthology.orchestra.run.smithi057.stdout: Birth: 2024-09-12 03:04:08.312586243 +0000 2024-09-12T03:06:00.201 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:06:00.201 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:06:00.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:06:00.291 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: from='client.25273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-mon[30361]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:06:00.101+0000 7fd85e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:06:00.101+0000 7fd85e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:06:00.101+0000 7fd85e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:06:00.101+0000 7fd85e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:06:00.292 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:06:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:06:00.101+0000 7fd85e000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:06:00.311 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:06:00.353 INFO:tasks.workunit.client.0.smithi057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:06:00.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:06:00.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:06:00.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:06:00.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:06:00.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: from='client.25273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:00.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:00 smithi204 ceph-mon[35013]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: from='client.25273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:00 smithi181 ceph-mon[35320]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:01.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:06:01.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:01.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-09-12T03:06:01.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:01.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: Updating smithi057:/etc/ceph/ceph.conf 2024-09-12T03:06:01.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-12T03:06:01.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:01 smithi057 ceph-mon[30361]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-12T03:06:01.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: Updating smithi057:/etc/ceph/ceph.conf 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-12T03:06:01.395 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:01 smithi204 ceph-mon[35013]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-12T03:06:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:06:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:06:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-09-12T03:06:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:01.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:01.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: Updating smithi057:/etc/ceph/ceph.conf 2024-09-12T03:06:01.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: Updating smithi181:/etc/ceph/ceph.conf 2024-09-12T03:06:01.455 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:01 smithi181 ceph-mon[35320]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-12T03:06:02.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:02 smithi057 ceph-mon[30361]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:02 smithi204 ceph-mon[35013]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi181:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:02.705 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:02 smithi181 ceph-mon[35320]: Updating smithi057:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: mgrmap e26: x(active, since 6s), standbys: y, z 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:03.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: mgrmap e26: x(active, since 6s), standbys: y, z 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:03.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-12T03:06:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: Updating smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:06:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: mgrmap e26: x(active, since 6s), standbys: y, z 2024-09-12T03:06:03.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:06:03.704 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:06:05.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:05 smithi057 ceph-mon[30361]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-09-12T03:06:05.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:05 smithi204 ceph-mon[35013]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-09-12T03:06:05.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:05 smithi181 ceph-mon[35320]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 171 KiB/s wr, 103 op/s 2024-09-12T03:06:07.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:07 smithi057 ceph-mon[30361]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-09-12T03:06:07.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:07 smithi204 ceph-mon[35013]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-09-12T03:06:07.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:07 smithi181 ceph-mon[35320]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-09-12T03:06:09.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:09 smithi057 ceph-mon[30361]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-12T03:06:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:09 smithi204 ceph-mon[35013]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-12T03:06:09.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:09 smithi181 ceph-mon[35320]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-09-12T03:06:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:11 smithi057 ceph-mon[30361]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:11 smithi204 ceph-mon[35013]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:11.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:11 smithi181 ceph-mon[35320]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 -- 172.21.15.57:0/3003502130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 msgr2=0x7fea78110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 --2- 172.21.15.57:0/3003502130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea78110bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea70009860 tx=0x7fea7002a340 comp rx=0 tx=0).stop 2024-09-12T03:06:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 -- 172.21.15.57:0/3003502130 shutdown_connections 2024-09-12T03:06:13.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 --2- 172.21.15.57:0/3003502130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea78110bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 --2- 172.21.15.57:0/3003502130 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea781048e0 0x7fea78108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 --2- 172.21.15.57:0/3003502130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea78103f30 0x7fea78104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 -- 172.21.15.57:0/3003502130 >> 172.21.15.57:0/3003502130 conn(0x7fea780ff980 msgr2=0x7fea78101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.330+0000 7fea7ce00640 1 -- 172.21.15.57:0/3003502130 shutdown_connections 2024-09-12T03:06:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.331+0000 7fea7ce00640 1 -- 172.21.15.57:0/3003502130 wait complete. 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.331+0000 7fea7ce00640 1 Processor -- start 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 -- start start 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea78103f30 0x7fea7819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 0x7fea7819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fea781140a0 con 0x7fea78103f30 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea78113f20 con 0x7fea7819ee20 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea7ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea78114220 con 0x7fea78109080 2024-09-12T03:06:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea75e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea78103f30 0x7fea7819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea76800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 0x7fea7819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56452/0 (socket says 172.21.15.57:56452) 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.332+0000 7fea6d400640 1 -- 172.21.15.57:0/2974914710 learned_addr learned my addr 172.21.15.57:0/2974914710 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 -- 172.21.15.57:0/2974914710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 msgr2=0x7fea7819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 -- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea78103f30 msgr2=0x7fea7819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea78103f30 0x7fea7819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 -- 172.21.15.57:0/2974914710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7819f910 con 0x7fea7819ee20 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea6d400640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea76800640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 0x7fea7819f2d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea7002a860 tx=0x7fea70033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea6f400640 1 -- 172.21.15.57:0/2974914710 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7003f020 con 0x7fea7819ee20 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7819fba0 con 0x7fea7819ee20 2024-09-12T03:06:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea6f400640 1 -- 172.21.15.57:0/2974914710 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea70030550 con 0x7fea7819ee20 2024-09-12T03:06:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.333+0000 7fea6f400640 1 -- 172.21.15.57:0/2974914710 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70037040 con 0x7fea7819ee20 2024-09-12T03:06:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.334+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea781a7360 con 0x7fea7819ee20 2024-09-12T03:06:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 Processor -- start 2024-09-12T03:06:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea6f400640 1 -- 172.21.15.57:0/2974914710 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7fea70042020 con 0x7fea7819ee20 2024-09-12T03:06:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 -- start start 2024-09-12T03:06:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a17413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a17414a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a1740a3100 con 0x55a17413d1e0 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a17404b1b0 con 0x55a17413db00 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.337+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a17409f0c0 con 0x55a1741483a0 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:32838/0 (socket says 172.21.15.57:32838) 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea6d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a17413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 -- 172.21.15.57:0/455306909 learned_addr learned my addr 172.21.15.57:0/455306909 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea6d400640 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a17414a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 -- 172.21.15.57:0/455306909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 msgr2=0x55a17414a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a17414a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 -- 172.21.15.57:0/455306909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 msgr2=0x55a17413d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a17413d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 -- 172.21.15.57:0/455306909 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a173db6210 con 0x55a17413db00 2024-09-12T03:06:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea6d400640 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a17413d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.338+0000 7fea75e00640 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea680114f0 tx=0x7fea680119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeec5ca303bcf0bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.339+0000 7fea6f400640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 0x7fea5c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.339+0000 7fea6f400640 1 -- 172.21.15.57:0/2974914710 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fea700ef8b0 con 0x7fea7819ee20 2024-09-12T03:06:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.339+0000 7fea6d400640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 0x7fea5c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.340+0000 7fea6d400640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 0x7fea5c0a5410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea600058c0 tx=0x7fea60009680 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.340+0000 7fea67400640 1 -- 172.21.15.57:0/455306909 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801e070 con 0x55a17413db00 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea67400640 1 -- 172.21.15.57:0/455306909 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6800fb00 con 0x55a17413db00 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea67400640 1 -- 172.21.15.57:0/455306909 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68019bf0 con 0x55a17413db00 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 -- 172.21.15.57:0/455306909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 msgr2=0x55a174147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea680114f0 tx=0x7fea680119b0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 -- 172.21.15.57:0/455306909 shutdown_connections 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a17414a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413db00 0x55a174147e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/455306909 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a17413d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 -- 172.21.15.57:0/455306909 >> 172.21.15.57:0/455306909 conn(0x55a174139750 msgr2=0x55a174139b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 -- 172.21.15.57:0/455306909 shutdown_connections 2024-09-12T03:06:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.341+0000 7fea7fdf9740 1 -- 172.21.15.57:0/455306909 wait complete. 2024-09-12T03:06:13.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.342+0000 7fea7fdf9740 1 -- start start 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.342+0000 7fea7fdf9740 1 Processor -- start 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a174145e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a17414af70 con 0x55a17413db00 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a17402c090 con 0x55a17413d1e0 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a17414b0f0 con 0x55a1741483a0 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea7fdf9740 10 client.0 _ll_register_callbacks cb 0x55a174080f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea6d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:32842/0 (socket says 172.21.15.57:32842) 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea6d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34840/0 (socket says 172.21.15.57:34840) 2024-09-12T03:06:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea76800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a174145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 -- 172.21.15.57:0/2874385057 learned_addr learned my addr 172.21.15.57:0/2874385057 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 -- 172.21.15.57:0/2874385057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 msgr2=0x55a174145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a174145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 -- 172.21.15.57:0/2874385057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 msgr2=0x55a1741458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.343+0000 7fea75e00640 1 -- 172.21.15.57:0/2874385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a173e82620 con 0x55a17413d1e0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea6d400640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea75e00640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea68011eb0 tx=0x7fea68019410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee065d0f8f799b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea5be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68029060 con 0x55a17413d1e0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6800fb00 con 0x55a17413d1e0 2024-09-12T03:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.344+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fea68004a40 con 0x55a17413d1e0 2024-09-12T03:06:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea68030050 con 0x55a17413d1e0 2024-09-12T03:06:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 10 client.25295 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a174156c00 con 0x55a17413d1e0 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6800fb00 con 0x55a17413d1e0 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea68005140 con 0x55a17413d1e0 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 10 client.25295 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 10 client.25295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 20 client.25295 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a174158070 0x55a17415a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 4 client.25295 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.345+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1740b0830 con 0x55a174158070 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.346+0000 7fea6d400640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a174158070 0x55a17415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.346+0000 7fea6d400640 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a174158070 0x55a17415a460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9221861be5537988 server_cookie=86a582841da39bd0 in_seq=0 out_seq=0 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.346+0000 7fea5be00640 10 client.25295 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea5be00640 1 -- 172.21.15.57:0/2874385057 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1740b0830 con 0x55a174158070 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea5be00640 10 client.25295 handle_command_reply: tid=0 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 client.25295 shutdown 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 msgr2=0x55a174145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea68011eb0 tx=0x7fea68019410 comp rx=0 tx=0).stop 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 shutdown_connections 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a174158070 0x55a17415a460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1741483a0 0x55a174145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea5be00640 0 client.25295 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413db00 0x55a1741458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/2874385057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a174145380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 >> 172.21.15.57:0/2874385057 conn(0x55a174139750 msgr2=0x55a17413f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 shutdown_connections 2024-09-12T03:06:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.347+0000 7fea7fdf9740 1 -- 172.21.15.57:0/2874385057 wait complete. 2024-09-12T03:06:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.348+0000 7fea7fdf9740 20 client.25295 trim_cache size 0 max 16384 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 Processor -- start 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 -- start start 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a1741471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1741483a0 0x55a17415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fea781b3aa0 con 0x55a1741483a0 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea781b1370 con 0x55a17413d1e0 2024-09-12T03:06:13.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.363+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea781af1c0 con 0x55a17413db00 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea76800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1741483a0 0x55a17415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a1741471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea6d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a1741471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:56476/0 (socket says 172.21.15.57:56476) 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:32852/0 (socket says 172.21.15.57:32852) 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 -- 172.21.15.57:0/85491508 learned_addr learned my addr 172.21.15.57:0/85491508 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 -- 172.21.15.57:0/85491508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 msgr2=0x55a1741471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a1741471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 -- 172.21.15.57:0/85491508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1741483a0 msgr2=0x55a17415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1741483a0 0x55a17415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 -- 172.21.15.57:0/85491508 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a174134d60 con 0x55a17413d1e0 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.364+0000 7fea75e00640 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea68009230 tx=0x7fea68005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed8bde5b3d4ac50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea66a00640 1 -- 172.21.15.57:0/85491508 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6801e040 con 0x55a17413d1e0 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea66a00640 1 -- 172.21.15.57:0/85491508 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea68003e50 con 0x55a17413d1e0 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea66a00640 1 -- 172.21.15.57:0/85491508 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680197c0 con 0x55a17413d1e0 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 -- 172.21.15.57:0/85491508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 msgr2=0x55a17413e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea68009230 tx=0x7fea68005700 comp rx=0 tx=0).stop 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 -- 172.21.15.57:0/85491508 shutdown_connections 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a1741483a0 0x55a17415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a1741471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/85491508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a17413d1e0 0x55a17413e380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 -- 172.21.15.57:0/85491508 >> 172.21.15.57:0/85491508 conn(0x55a174139750 msgr2=0x55a174146ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 -- 172.21.15.57:0/85491508 shutdown_connections 2024-09-12T03:06:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.365+0000 7fea7fdf9740 1 -- 172.21.15.57:0/85491508 wait complete. 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.366+0000 7fea7fdf9740 1 -- start start 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.366+0000 7fea7fdf9740 1 Processor -- start 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.366+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a174150b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 0x55a174141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fea781af830 con 0x55a17413d1e0 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fea781b0ca0 con 0x55a1741483a0 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea781aff00 con 0x55a17413db00 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea7fdf9740 10 client.0 _ll_register_callbacks cb 0x7fea6801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea75e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea75e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:34862/0 (socket says 172.21.15.57:34862) 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea75e00640 1 -- 172.21.15.57:0/1389059783 learned_addr learned my addr 172.21.15.57:0/1389059783 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 0x55a174141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea6d400640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a174150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 -- 172.21.15.57:0/1389059783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 msgr2=0x55a174150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a174150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 -- 172.21.15.57:0/1389059783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 msgr2=0x55a174150600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea76800640 1 -- 172.21.15.57:0/1389059783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a174142570 con 0x55a1741483a0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.367+0000 7fea75e00640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea76800640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 0x55a174141d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea700ecb20 tx=0x7fea700bb4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d811c2ed8e5ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea700d3020 con 0x55a1741483a0 2024-09-12T03:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fea700bbb50 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea68003e50 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea700a62b0 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 10 client.25301 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1741c2770 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea68003e50 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fea7003a030 con 0x55a1741483a0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 10 client.25301 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 20 client.25301 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.368+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a1741c2d00 0x55a1741c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.369+0000 7fea7fdf9740 4 client.25301 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.369+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1740aab80 con 0x55a1741c2d00 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.369+0000 7fea75e00640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a1741c2d00 0x55a1741c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.369+0000 7fea75e00640 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a1741c2d00 0x55a1741c50f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f455c4ad04732dc5 server_cookie=da98872e3d474fa8 in_seq=0 out_seq=0 2024-09-12T03:06:13.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.369+0000 7fea5be00640 10 client.25301 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.382+0000 7fea5be00640 1 -- 172.21.15.57:0/1389059783 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1740aab80 con 0x55a1741c2d00 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.382+0000 7fea5be00640 10 client.25301 handle_command_reply: tid=0 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.382+0000 7fea7fdf9740 1 client.25301 shutdown 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.382+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 msgr2=0x55a174141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 0x55a174141d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea700ecb20 tx=0x7fea700bb4f0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 shutdown_connections 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a1741c2d00 0x55a1741c50f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a1741483a0 0x55a174141d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea5be00640 0 client.25301 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a17413db00 0x55a174150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 --2- 172.21.15.57:0/1389059783 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a17413d1e0 0x55a174150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 >> 172.21.15.57:0/1389059783 conn(0x55a174139750 msgr2=0x55a17413f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 shutdown_connections 2024-09-12T03:06:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 1 -- 172.21.15.57:0/1389059783 wait complete. 2024-09-12T03:06:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.383+0000 7fea7fdf9740 20 client.25301 trim_cache size 0 max 16384 2024-09-12T03:06:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:06:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:06:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:06:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:06:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 msgr2=0x7fea5c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 0x7fea5c0a5410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fea600058c0 tx=0x7fea60009680 comp rx=0 tx=0).stop 2024-09-12T03:06:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 msgr2=0x7fea7819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 0x7fea7819f2d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea7002a860 tx=0x7fea70033ae0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 shutdown_connections 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea7819ee20 0x7fea7819f2d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea78109080 0x7fea7819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fea5c0a2f50 0x7fea5c0a5410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 --2- 172.21.15.57:0/2974914710 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fea78103f30 0x7fea7819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 >> 172.21.15.57:0/2974914710 conn(0x7fea780ff980 msgr2=0x7fea78100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 shutdown_connections 2024-09-12T03:06:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:13.384+0000 7fea7ce00640 1 -- 172.21.15.57:0/2974914710 wait complete. 2024-09-12T03:06:13.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:13 smithi057 ceph-mon[30361]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:13 smithi204 ceph-mon[35013]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:13.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:13 smithi181 ceph-mon[35320]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 100 KiB/s wr, 63 op/s 2024-09-12T03:06:15.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:15 smithi057 ceph-mon[30361]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-09-12T03:06:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:15 smithi204 ceph-mon[35013]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-09-12T03:06:15.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:15 smithi181 ceph-mon[35320]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-09-12T03:06:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:17 smithi057 ceph-mon[30361]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:06:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:17 smithi204 ceph-mon[35013]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:06:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:17 smithi181 ceph-mon[35320]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:06:19.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:19 smithi057 ceph-mon[30361]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T03:06:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:19 smithi204 ceph-mon[35013]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T03:06:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:19 smithi181 ceph-mon[35320]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 KiB/s wr, 12 op/s 2024-09-12T03:06:21.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:21 smithi057 ceph-mon[30361]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:21.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:21 smithi204 ceph-mon[35013]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:21 smithi181 ceph-mon[35320]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:23.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:23 smithi057 ceph-mon[30361]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-12T03:06:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:23 smithi204 ceph-mon[35013]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-12T03:06:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:23 smithi181 ceph-mon[35320]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-12T03:06:25.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:25 smithi204 ceph-mon[35013]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-12T03:06:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:25 smithi181 ceph-mon[35320]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-12T03:06:25.756 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:25 smithi057 ceph-mon[30361]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-12T03:06:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 -- 172.21.15.57:0/1189855951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6010102250 msgr2=0x7f601010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 --2- 172.21.15.57:0/1189855951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6010102250 0x7f601010e780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6008009cd0 tx=0x7f600802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 -- 172.21.15.57:0/1189855951 shutdown_connections 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 --2- 172.21.15.57:0/1189855951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6010102250 0x7f601010e780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 --2- 172.21.15.57:0/1189855951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60101018b0 0x7f6010101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 --2- 172.21.15.57:0/1189855951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 0x7f6010107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 -- 172.21.15.57:0/1189855951 >> 172.21.15.57:0/1189855951 conn(0x7f60100fd5d0 msgr2=0x7f60100ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 -- 172.21.15.57:0/1189855951 shutdown_connections 2024-09-12T03:06:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.507+0000 7f600f200640 1 -- 172.21.15.57:0/1189855951 wait complete. 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.508+0000 7f600f200640 1 Processor -- start 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.508+0000 7f600f200640 1 -- start start 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 0x7f601019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111d00 con 0x7f60101018b0 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111b80 con 0x7f6010102250 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111e80 con 0x7f60101078c0 2024-09-12T03:06:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45064/0 (socket says 172.21.15.57:45064) 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:36576/0 (socket says 172.21.15.57:36576) 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600de00640 1 -- 172.21.15.57:0/3527366785 learned_addr learned my addr 172.21.15.57:0/3527366785 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.509+0000 7f600e800640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 0x7f601019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 -- 172.21.15.57:0/3527366785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 msgr2=0x7f601019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 0x7f601019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 -- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 msgr2=0x7f601019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 -- 172.21.15.57:0/3527366785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601019f3e0 con 0x7f6010102250 2024-09-12T03:06:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600de00640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600d400640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e9f0 tx=0x7f5ffc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f6006a00640 1 -- 172.21.15.57:0/3527366785 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00ccc0 con 0x7f6010102250 2024-09-12T03:06:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f6006a00640 1 -- 172.21.15.57:0/3527366785 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ffc010910 con 0x7f6010102250 2024-09-12T03:06:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f6006a00640 1 -- 172.21.15.57:0/3527366785 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc018960 con 0x7f6010102250 2024-09-12T03:06:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.510+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601019f6d0 con 0x7f6010102250 2024-09-12T03:06:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.511+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a6f60 con 0x7f6010102250 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 Processor -- start 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 -- start start 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc3cd0 0x557bdbbce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbbd0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x557bdbb29100 con 0x557bdbbce570 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557bdbad11b0 con 0x557bdbbc33b0 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f60179d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557bdbb250c0 con 0x557bdbbc3cd0 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f600e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f600e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:36590/0 (socket says 172.21.15.57:36590) 2024-09-12T03:06:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f600e800640 1 -- 172.21.15.57:0/2756300573 learned_addr learned my addr 172.21.15.57:0/2756300573 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:26.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.514+0000 7f600de00640 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc3cd0 0x557bdbbce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 -- 172.21.15.57:0/2756300573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc3cd0 msgr2=0x557bdbbce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc3cd0 0x557bdbbce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 -- 172.21.15.57:0/2756300573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 msgr2=0x557bdbbd0960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbbd0960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 -- 172.21.15.57:0/2756300573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bdb83c210 con 0x557bdbbc33b0 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f6006a00640 1 -- 172.21.15.57:0/3527366785 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7f5ffc010ac0 con 0x7f6010102250 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.515+0000 7f600e800640 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6008009cd0 tx=0x7f6008009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4da7f60165abf0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.516+0000 7f5ffb000640 1 -- 172.21.15.57:0/2756300573 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008048070 con 0x557bdbbc33b0 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.516+0000 7f5ffb000640 1 -- 172.21.15.57:0/2756300573 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008036070 con 0x557bdbbc33b0 2024-09-12T03:06:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.516+0000 7f5ffb000640 1 -- 172.21.15.57:0/2756300573 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008004440 con 0x557bdbbc33b0 2024-09-12T03:06:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.516+0000 7f6006a00640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 0x7f5ff40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 -- 172.21.15.57:0/2756300573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 msgr2=0x557bdbbc3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6008009cd0 tx=0x7f6008009a20 comp rx=0 tx=0).stop 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f6006a00640 1 -- 172.21.15.57:0/3527366785 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5ffc0cfdf0 con 0x7f6010102250 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 -- 172.21.15.57:0/2756300573 shutdown_connections 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbbd0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc3cd0 0x557bdbbce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 --2- 172.21.15.57:0/2756300573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbc3790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 -- 172.21.15.57:0/2756300573 >> 172.21.15.57:0/2756300573 conn(0x557bdbbbf920 msgr2=0x557bdbbbfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f600de00640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 0x7f5ff40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 -- 172.21.15.57:0/2756300573 shutdown_connections 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.517+0000 7f60179d6740 1 -- 172.21.15.57:0/2756300573 wait complete. 2024-09-12T03:06:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.518+0000 7f600de00640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 0x7f5ff40a53e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x557bdbbc0780 tx=0x7f6000005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.518+0000 7f60179d6740 1 -- start start 2024-09-12T03:06:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.518+0000 7f60179d6740 1 Processor -- start 2024-09-12T03:06:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbcb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbce570 0x557bdbbcc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x557bdbbd0ff0 con 0x557bdbbc3cd0 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557bdbab2090 con 0x557bdbbc33b0 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557bdbbd1170 con 0x557bdbbce570 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f60179d6740 10 client.0 _ll_register_callbacks cb 0x557bdbb06f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f600de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbcb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f600d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f600de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbcb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:36604/0 (socket says 172.21.15.57:36604) 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.519+0000 7f600de00640 1 -- 172.21.15.57:0/560930752 learned_addr learned my addr 172.21.15.57:0/560930752 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600e800640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbce570 0x557bdbbcc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 -- 172.21.15.57:0/560930752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbce570 msgr2=0x557bdbbcc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbce570 0x557bdbbcc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 -- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 msgr2=0x557bdbbcb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbcb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 -- 172.21.15.57:0/560930752 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bdb908620 con 0x557bdbbc3cd0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f600d400640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcbb90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0ccf50 tx=0x7f5ffc098180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52d3ddbc5bee0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f5fef400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.520+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0a4990 con 0x557bdbbc3cd0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5ffc0cd260 con 0x557bdbbc3cd0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6008036070 con 0x557bdbbc3cd0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc0cd460 con 0x557bdbbc3cd0 2024-09-12T03:06:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f60179d6740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557bdbbdce90 con 0x557bdbbc3cd0 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008036070 con 0x557bdbbc3cd0 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ffc097d00 con 0x557bdbbc3cd0 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f60179d6740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.521+0000 7f60179d6740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.522+0000 7f60179d6740 20 client.15270 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.522+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbbde300 0x557bdbbe06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.522+0000 7f60179d6740 4 client.15270 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.522+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bdbb36830 con 0x557bdbbde300 2024-09-12T03:06:26.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.522+0000 7f600de00640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbbde300 0x557bdbbe06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.523+0000 7f600de00640 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbbde300 0x557bdbbe06f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab7464901f51cf0 server_cookie=e11dc7494f99af26 in_seq=0 out_seq=0 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.523+0000 7f5fef400640 10 client.15270 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.523+0000 7f5fef400640 1 -- 172.21.15.57:0/560930752 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bdbb36830 con 0x557bdbbde300 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.523+0000 7f5fef400640 10 client.15270 handle_command_reply: tid=0 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.523+0000 7f60179d6740 1 client.15270 shutdown 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 msgr2=0x557bdbbcbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcbb90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0ccf50 tx=0x7f5ffc098180 comp rx=0 tx=0).stop 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 shutdown_connections 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbbde300 0x557bdbbe06f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbce570 0x557bdbbcc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcbb90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 --2- 172.21.15.57:0/560930752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc33b0 0x557bdbbcb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 >> 172.21.15.57:0/560930752 conn(0x557bdbbbf920 msgr2=0x557bdbbc5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f5fef400640 0 client.15270 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 shutdown_connections 2024-09-12T03:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 1 -- 172.21.15.57:0/560930752 wait complete. 2024-09-12T03:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.524+0000 7f60179d6740 20 client.15270 trim_cache size 0 max 16384 2024-09-12T03:06:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 Processor -- start 2024-09-12T03:06:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 -- start start 2024-09-12T03:06:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbc5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 0x557bdbbe5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60101b39a0 con 0x557bdbbc3cd0 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60101b1270 con 0x557bdbbce570 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f60179d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60101af0c0 con 0x557bdbbc33b0 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.555+0000 7f600d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 0x557bdbbe5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600d400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45084/0 (socket says 172.21.15.57:45084) 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600d400640 1 -- 172.21.15.57:0/1839650178 learned_addr learned my addr 172.21.15.57:0/1839650178 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600de00640 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbc5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 -- 172.21.15.57:0/1839650178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 msgr2=0x557bdbbc5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbc5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 -- 172.21.15.57:0/1839650178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 msgr2=0x557bdbbcd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 -- 172.21.15.57:0/1839650178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bdbbe0cd0 con 0x557bdbbce570 2024-09-12T03:06:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600d400640 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.556+0000 7f600e800640 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 0x557bdbbe5f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f600802ae00 tx=0x7f6008003f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec94a39e72d912f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f5ffa600640 1 -- 172.21.15.57:0/1839650178 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008053070 con 0x557bdbbce570 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f5ffa600640 1 -- 172.21.15.57:0/1839650178 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008036040 con 0x557bdbbce570 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f5ffa600640 1 -- 172.21.15.57:0/1839650178 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008048040 con 0x557bdbbce570 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f60179d6740 1 -- 172.21.15.57:0/1839650178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 msgr2=0x557bdbbe5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f60179d6740 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 0x557bdbbe5f60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f600802ae00 tx=0x7f6008003f60 comp rx=0 tx=0).stop 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.557+0000 7f60179d6740 1 -- 172.21.15.57:0/1839650178 shutdown_connections 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbce570 0x557bdbbe5f60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbc3cd0 0x557bdbbcd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 --2- 172.21.15.57:0/1839650178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbc5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 -- 172.21.15.57:0/1839650178 >> 172.21.15.57:0/1839650178 conn(0x557bdbbbf920 msgr2=0x557bdbbccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 -- 172.21.15.57:0/1839650178 shutdown_connections 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 -- 172.21.15.57:0/1839650178 wait complete. 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 -- start start 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.558+0000 7f60179d6740 1 Processor -- start 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbd1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 0x557bdbbd2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f60101af730 con 0x557bdbbce570 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f60101b0ba0 con 0x557bdbbc3cd0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60101afe00 con 0x557bdbbc33b0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f60179d6740 10 client.0 _ll_register_callbacks cb 0x7f5ffc00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbd1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45096/0 (socket says 172.21.15.57:45096) 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 -- 172.21.15.57:0/314298855 learned_addr learned my addr 172.21.15.57:0/314298855 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbd1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:57694/0 (socket says 172.21.15.57:57694) 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600d400640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 0x557bdbbd2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 -- 172.21.15.57:0/314298855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 msgr2=0x557bdbbd1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbd1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 -- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 msgr2=0x557bdbbd2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 0x557bdbbd2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600e800640 1 -- 172.21.15.57:0/314298855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bdbc41550 con 0x557bdbbce570 2024-09-12T03:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.559+0000 7f600d400640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 0x557bdbbd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.560+0000 7f600e800640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6008003cc0 tx=0x7f6008003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0afe81956a18a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.560+0000 7f5fef400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.560+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008048070 con 0x557bdbbce570 2024-09-12T03:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.560+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6008036040 con 0x557bdbbce570 2024-09-12T03:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.560+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f6008031ac0 con 0x557bdbbce570 2024-09-12T03:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6008053040 con 0x557bdbbce570 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557bdbc488e0 con 0x557bdbbce570 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6008036040 con 0x557bdbbce570 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6008042030 con 0x557bdbbce570 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 20 client.15276 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbc48b10 0x557bdbc4af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 4 client.15276 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.561+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bdbb30b80 con 0x557bdbc48b10 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.562+0000 7f600de00640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbc48b10 0x557bdbc4af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.563+0000 7f5fef400640 10 client.15276 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.563+0000 7f600de00640 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbc48b10 0x557bdbc4af00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb40bdb561ce17f1 server_cookie=675d8d94407f5b0d in_seq=0 out_seq=0 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f5fef400640 1 -- 172.21.15.57:0/314298855 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bdbb30b80 con 0x557bdbc48b10 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f5fef400640 10 client.15276 handle_command_reply: tid=0 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 client.15276 shutdown 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 msgr2=0x557bdbc40f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6008003cc0 tx=0x7f6008003cf0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 shutdown_connections 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x557bdbc48b10 0x557bdbc4af00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557bdbbce570 0x557bdbc40f60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557bdbbc3cd0 0x557bdbbd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 --2- 172.21.15.57:0/314298855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bdbbc33b0 0x557bdbbd1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f5fef400640 0 client.15276 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 >> 172.21.15.57:0/314298855 conn(0x557bdbbbf920 msgr2=0x557bdbbbe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 shutdown_connections 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 1 -- 172.21.15.57:0/314298855 wait complete. 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.571+0000 7f60179d6740 20 client.15276 trim_cache size 0 max 16384 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 msgr2=0x7f5ff40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 0x7f5ff40a53e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x557bdbbc0780 tx=0x7f6000005f90 comp rx=0 tx=0).stop 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 msgr2=0x7f601019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00e9f0 tx=0x7f5ffc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 shutdown_connections 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60101078c0 0x7f601019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6010102250 0x7f601019a930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ff40a2f20 0x7f5ff40a53e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 --2- 172.21.15.57:0/3527366785 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f60101018b0 0x7f601019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 >> 172.21.15.57:0/3527366785 conn(0x7f60100fd5d0 msgr2=0x7f601010ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.572+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 shutdown_connections 2024-09-12T03:06:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:26.573+0000 7f600f200640 1 -- 172.21.15.57:0/3527366785 wait complete. 2024-09-12T03:06:27.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:27 smithi204 ceph-mon[35013]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:27 smithi181 ceph-mon[35320]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:27.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:27 smithi057 ceph-mon[30361]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-12T03:06:29.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:29 smithi204 ceph-mon[35013]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T03:06:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:29 smithi181 ceph-mon[35320]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T03:06:29.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:29 smithi057 ceph-mon[30361]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T03:06:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:31 smithi181 ceph-mon[35320]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:31.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:31 smithi057 ceph-mon[30361]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:31.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:31 smithi204 ceph-mon[35013]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:33 smithi181 ceph-mon[35320]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:33.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:33 smithi057 ceph-mon[30361]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:33 smithi204 ceph-mon[35013]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:35 smithi181 ceph-mon[35320]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:35.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:35 smithi057 ceph-mon[30361]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:35.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:35 smithi204 ceph-mon[35013]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-12T03:06:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:37 smithi181 ceph-mon[35320]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:37.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:37 smithi057 ceph-mon[30361]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:37 smithi204 ceph-mon[35013]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-12T03:06:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:39 smithi181 ceph-mon[35320]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 -- 172.21.15.57:0/1003676420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0250109080 msgr2=0x7f0250110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 --2- 172.21.15.57:0/1003676420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0250109080 0x7f0250110bd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0248009be0 tx=0x7f024802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 -- 172.21.15.57:0/1003676420 shutdown_connections 2024-09-12T03:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 --2- 172.21.15.57:0/1003676420 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0250109080 0x7f0250110bd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 --2- 172.21.15.57:0/1003676420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02501048e0 0x7f0250108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 --2- 172.21.15.57:0/1003676420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250103f30 0x7f0250104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.755+0000 7f0254e00640 1 -- 172.21.15.57:0/1003676420 >> 172.21.15.57:0/1003676420 conn(0x7f02500ff980 msgr2=0x7f0250101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.756+0000 7f0254e00640 1 -- 172.21.15.57:0/1003676420 shutdown_connections 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.756+0000 7f0254e00640 1 -- 172.21.15.57:0/1003676420 wait complete. 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.757+0000 7f0254e00640 1 Processor -- start 2024-09-12T03:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.757+0000 7f0254e00640 1 -- start start 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250109080 0x7f025019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 0x7f025019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0250114090 con 0x7f025019ede0 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0250113f10 con 0x7f0250103f30 2024-09-12T03:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f0254e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0250114210 con 0x7f0250109080 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f024de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f024e800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 0x7f025019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f024de00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41342/0 (socket says 172.21.15.57:41342) 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f024de00640 1 -- 172.21.15.57:0/1961102666 learned_addr learned my addr 172.21.15.57:0/1961102666 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.758+0000 7f024d400640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250109080 0x7f025019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 -- 172.21.15.57:0/1961102666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250109080 msgr2=0x7f025019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250109080 0x7f025019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 -- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 msgr2=0x7f025019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 0x7f025019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 -- 172.21.15.57:0/1961102666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025019f8d0 con 0x7f0250103f30 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024e800640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 0x7f025019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f024de00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f024000e9f0 tx=0x7f024000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f0246a00640 1 -- 172.21.15.57:0/1961102666 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024000cd80 con 0x7f0250103f30 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f0246a00640 1 -- 172.21.15.57:0/1961102666 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0240005500 con 0x7f0250103f30 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.759+0000 7f0246a00640 1 -- 172.21.15.57:0/1961102666 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240010690 con 0x7f0250103f30 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.760+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025019fbc0 con 0x7f0250103f30 2024-09-12T03:06:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.760+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a7360 con 0x7f0250103f30 2024-09-12T03:06:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.764+0000 7f0257dd8740 1 Processor -- start 2024-09-12T03:06:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.764+0000 7f0257dd8740 1 -- start start 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0246a00640 1 -- 172.21.15.57:0/1961102666 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7f02400046c0 con 0x7f0250103f30 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc0d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0db00 0x5612dfc17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5612dfb73100 con 0x5612dfc0db00 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5612dfb1b1b0 con 0x5612dfc183a0 2024-09-12T03:06:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f0257dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5612dfb6f0c0 con 0x5612dfc0d1e0 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f024de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0db00 0x5612dfc17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f024d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f024d400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41356/0 (socket says 172.21.15.57:41356) 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.765+0000 7f024d400640 1 -- 172.21.15.57:0/2036069185 learned_addr learned my addr 172.21.15.57:0/2036069185 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024e800640 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc0d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 -- 172.21.15.57:0/2036069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 msgr2=0x5612dfc0d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc0d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 -- 172.21.15.57:0/2036069185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0db00 msgr2=0x5612dfc17e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0db00 0x5612dfc17e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 -- 172.21.15.57:0/2036069185 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612df886210 con 0x5612dfc183a0 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f024d400640 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f023c009c60 tx=0x7f023c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472f84b864a81673 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.766+0000 7f0246a00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 0x7f02340a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.767+0000 7f023b400640 1 -- 172.21.15.57:0/2036069185 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c009e10 con 0x5612dfc183a0 2024-09-12T03:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.767+0000 7f023b400640 1 -- 172.21.15.57:0/2036069185 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023c0035f0 con 0x5612dfc183a0 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.767+0000 7f023b400640 1 -- 172.21.15.57:0/2036069185 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c019650 con 0x5612dfc183a0 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.767+0000 7f0246a00640 1 -- 172.21.15.57:0/1961102666 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f02400cff70 con 0x7f0250103f30 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.767+0000 7f024e800640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 0x7f02340a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 -- 172.21.15.57:0/2036069185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 msgr2=0x5612dfc1a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f023c009c60 tx=0x7f023c011810 comp rx=0 tx=0).stop 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 -- 172.21.15.57:0/2036069185 shutdown_connections 2024-09-12T03:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc1a790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f024e800640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 0x7f02340a5410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0248004af0 tx=0x7f0248004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0db00 0x5612dfc17e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2036069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc0d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 -- 172.21.15.57:0/2036069185 >> 172.21.15.57:0/2036069185 conn(0x5612dfc09750 msgr2=0x5612dfc09b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.768+0000 7f0257dd8740 1 -- 172.21.15.57:0/2036069185 shutdown_connections 2024-09-12T03:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.769+0000 7f0257dd8740 1 -- 172.21.15.57:0/2036069185 wait complete. 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.769+0000 7f0257dd8740 1 -- start start 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 Processor -- start 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc15840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 0x5612dfc15d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5612dfc1af70 con 0x5612dfc0d1e0 2024-09-12T03:06:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5612dfafc090 con 0x5612dfc0db00 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.770+0000 7f0257dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5612dfc1b0f0 con 0x5612dfc183a0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f0257dd8740 10 client.0 _ll_register_callbacks cb 0x5612dfb50f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 0x5612dfc15d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024de00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36994/0 (socket says 172.21.15.57:36994) 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024de00640 1 -- 172.21.15.57:0/497454227 learned_addr learned my addr 172.21.15.57:0/497454227 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024d400640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc15840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 -- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 msgr2=0x5612dfc15840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc15840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 -- 172.21.15.57:0/497454227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 msgr2=0x5612dfc15300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024e800640 1 -- 172.21.15.57:0/497454227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612df952620 con 0x5612dfc183a0 2024-09-12T03:06:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.771+0000 7f024de00640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.772+0000 7f024e800640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 0x5612dfc15d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f024802abe0 tx=0x7f024802a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8dc46f2282523f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.772+0000 7f022fe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.772+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248050070 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.772+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f023c0035f0 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.772+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f0248004040 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.773+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f024805c050 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.773+0000 7f0257dd8740 10 client.25339 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.773+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612dfc26b80 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.773+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023c0035f0 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f024803c040 con 0x5612dfc183a0 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 10 client.25339 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 10 client.25339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 20 client.25339 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc27ff0 0x5612dfc2a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 4 client.25339 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612dfb80830 con 0x5612dfc27ff0 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.774+0000 7f024de00640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc27ff0 0x5612dfc2a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.775+0000 7f024de00640 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc27ff0 0x5612dfc2a400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2330af70162d48 server_cookie=6e1e20163f22418e in_seq=0 out_seq=0 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.775+0000 7f022fe00640 10 client.25339 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f022fe00640 1 -- 172.21.15.57:0/497454227 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612dfb80830 con 0x5612dfc27ff0 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f022fe00640 10 client.25339 handle_command_reply: tid=0 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 client.25339 shutdown 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 msgr2=0x5612dfc15d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 0x5612dfc15d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f024802abe0 tx=0x7f024802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 shutdown_connections 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc27ff0 0x5612dfc2a400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc183a0 0x5612dfc15d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc15840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 --2- 172.21.15.57:0/497454227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc15300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 >> 172.21.15.57:0/497454227 conn(0x5612dfc09750 msgr2=0x5612dfc0f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f022fe00640 0 client.25339 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 shutdown_connections 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.776+0000 7f0257dd8740 1 -- 172.21.15.57:0/497454227 wait complete. 2024-09-12T03:06:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.777+0000 7f0257dd8740 20 client.25339 trim_cache size 0 max 16384 2024-09-12T03:06:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.806+0000 7f0257dd8740 1 Processor -- start 2024-09-12T03:06:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 -- start start 2024-09-12T03:06:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc183a0 0x5612dfc0ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b3aa0 con 0x5612dfc183a0 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b1370 con 0x5612dfc0db00 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f0257dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02501af1c0 con 0x5612dfc0d1e0 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f024de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f024de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43548/0 (socket says 172.21.15.57:43548) 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.807+0000 7f024de00640 1 -- 172.21.15.57:0/3536362275 learned_addr learned my addr 172.21.15.57:0/3536362275 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024e800640 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc183a0 0x5612dfc0ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024d400640 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 -- 172.21.15.57:0/3536362275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 msgr2=0x5612dfc302a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc302a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 -- 172.21.15.57:0/3536362275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc183a0 msgr2=0x5612dfc0ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc183a0 0x5612dfc0ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 -- 172.21.15.57:0/3536362275 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dfc0a8e0 con 0x5612dfc0d1e0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f024de00640 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02400cd210 tx=0x7f02400d2400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff2470e7668ce5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f023aa00640 1 -- 172.21.15.57:0/3536362275 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024009b9e0 con 0x5612dfc0d1e0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f023aa00640 1 -- 172.21.15.57:0/3536362275 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024009c5a0 con 0x5612dfc0d1e0 2024-09-12T03:06:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.808+0000 7f023aa00640 1 -- 172.21.15.57:0/3536362275 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024009b800 con 0x5612dfc0d1e0 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 -- 172.21.15.57:0/3536362275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 msgr2=0x5612dfc2fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02400cd210 tx=0x7f02400d2400 comp rx=0 tx=0).stop 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 -- 172.21.15.57:0/3536362275 shutdown_connections 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc183a0 0x5612dfc0ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc0db00 0x5612dfc302a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 --2- 172.21.15.57:0/3536362275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0d1e0 0x5612dfc2fd60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 -- 172.21.15.57:0/3536362275 >> 172.21.15.57:0/3536362275 conn(0x5612dfc09750 msgr2=0x5612dfc10680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.809+0000 7f0257dd8740 1 -- 172.21.15.57:0/3536362275 shutdown_connections 2024-09-12T03:06:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.810+0000 7f0257dd8740 1 -- 172.21.15.57:0/3536362275 wait complete. 2024-09-12T03:06:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.810+0000 7f0257dd8740 1 -- start start 2024-09-12T03:06:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.810+0000 7f0257dd8740 1 Processor -- start 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0db00 0x5612dfc121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f02501af830 con 0x5612dfc0d1e0 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02501b0ca0 con 0x5612dfc183a0 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02501aff00 con 0x5612dfc0db00 2024-09-12T03:06:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f0257dd8740 10 client.0 _ll_register_callbacks cb 0x7f0248043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f024e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f024d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0db00 0x5612dfc121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f024e800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41372/0 (socket says 172.21.15.57:41372) 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f024e800640 1 -- 172.21.15.57:0/2003429875 learned_addr learned my addr 172.21.15.57:0/2003429875 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.811+0000 7f024de00640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 -- 172.21.15.57:0/2003429875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0db00 msgr2=0x5612dfc121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0db00 0x5612dfc121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 -- 172.21.15.57:0/2003429875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 msgr2=0x5612dfc20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 -- 172.21.15.57:0/2003429875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dfc91d30 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024de00640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f024e800640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0248031ed0 tx=0x7f0248009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53aca96e0a9da039 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f022fe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248050040 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f024804c8a0 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.812+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024009c5a0 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f024806e050 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 10 client.25337 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612dfc92f00 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f024009c5a0 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f024803c040 con 0x5612dfc183a0 2024-09-12T03:06:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 10 client.25337 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 20 client.25337 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc93640 0x5612dfc95a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 4 client.25337 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.813+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612dfb7ab80 con 0x5612dfc93640 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.814+0000 7f024de00640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc93640 0x5612dfc95a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.814+0000 7f024de00640 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc93640 0x5612dfc95a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985143fc1d4a02a0 server_cookie=c9d1c6962f4588dd in_seq=0 out_seq=0 2024-09-12T03:06:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.814+0000 7f022fe00640 10 client.25337 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.822+0000 7f022fe00640 1 -- 172.21.15.57:0/2003429875 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612dfb7ab80 con 0x5612dfc93640 2024-09-12T03:06:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.822+0000 7f022fe00640 10 client.25337 handle_command_reply: tid=0 2024-09-12T03:06:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.822+0000 7f0257dd8740 1 client.25337 shutdown 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 msgr2=0x5612dfc12930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0248031ed0 tx=0x7f0248009ba0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 shutdown_connections 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5612dfc93640 0x5612dfc95a60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5612dfc183a0 0x5612dfc12930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612dfc0db00 0x5612dfc121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f022fe00640 0 client.25337 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 --2- 172.21.15.57:0/2003429875 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5612dfc0d1e0 0x5612dfc20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 >> 172.21.15.57:0/2003429875 conn(0x5612dfc09750 msgr2=0x5612dfc0ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 shutdown_connections 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 1 -- 172.21.15.57:0/2003429875 wait complete. 2024-09-12T03:06:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.823+0000 7f0257dd8740 20 client.25337 trim_cache size 0 max 16384 2024-09-12T03:06:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:06:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:06:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:06:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 msgr2=0x7f02340a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 0x7f02340a5410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0248004af0 tx=0x7f0248004a40 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 msgr2=0x7f025019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f024000e9f0 tx=0x7f024000edf0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 shutdown_connections 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02340a2f50 0x7f02340a5410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f025019ede0 0x7f025019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0250109080 0x7f025019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 --2- 172.21.15.57:0/1961102666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0250103f30 0x7f025019a520 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 >> 172.21.15.57:0/1961102666 conn(0x7f02500ff980 msgr2=0x7f0250100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.824+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 shutdown_connections 2024-09-12T03:06:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:39.825+0000 7f0254e00640 1 -- 172.21.15.57:0/1961102666 wait complete. 2024-09-12T03:06:39.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:39 smithi057 ceph-mon[30361]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:06:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:39 smithi204 ceph-mon[35013]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-09-12T03:06:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:41 smithi181 ceph-mon[35320]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:06:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:41 smithi057 ceph-mon[30361]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:06:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:41 smithi204 ceph-mon[35013]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-12T03:06:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:43 smithi181 ceph-mon[35320]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:43.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:43 smithi057 ceph-mon[30361]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:43 smithi204 ceph-mon[35013]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:45.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:45 smithi181 ceph-mon[35320]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:45 smithi057 ceph-mon[30361]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:45 smithi204 ceph-mon[35013]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:47 smithi181 ceph-mon[35320]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:47.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:47 smithi057 ceph-mon[30361]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:47.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:47 smithi204 ceph-mon[35013]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:06:49.696 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:49 smithi204 ceph-mon[35013]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:49 smithi181 ceph-mon[35320]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:49.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:49 smithi057 ceph-mon[30361]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-12T03:06:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:51 smithi181 ceph-mon[35320]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:06:51.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:51 smithi057 ceph-mon[30361]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:06:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:51 smithi204 ceph-mon[35013]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-12T03:06:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 -- 172.21.15.57:0/2045800223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e81018b0 msgr2=0x7f11e8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 --2- 172.21.15.57:0/2045800223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e81018b0 0x7f11e8101d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009930 tx=0x7f11d402a420 comp rx=0 tx=0).stop 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 -- 172.21.15.57:0/2045800223 shutdown_connections 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 --2- 172.21.15.57:0/2045800223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e8102250 0x7f11e810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 --2- 172.21.15.57:0/2045800223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e81018b0 0x7f11e8101d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 --2- 172.21.15.57:0/2045800223 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 0x7f11e8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.015+0000 7f11eca00640 1 -- 172.21.15.57:0/2045800223 >> 172.21.15.57:0/2045800223 conn(0x7f11e80fd5d0 msgr2=0x7f11e80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.016+0000 7f11eca00640 1 -- 172.21.15.57:0/2045800223 shutdown_connections 2024-09-12T03:06:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.016+0000 7f11eca00640 1 -- 172.21.15.57:0/2045800223 wait complete. 2024-09-12T03:06:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.017+0000 7f11eca00640 1 Processor -- start 2024-09-12T03:06:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.017+0000 7f11eca00640 1 -- start start 2024-09-12T03:06:53.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8102250 0x7f11e810bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 0x7f11e810c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80760e0 con 0x7f11e81078c0 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8075f60 con 0x7f11e81018b0 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11eca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8076260 con 0x7f11e8102250 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11e7400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51390/0 (socket says 172.21.15.57:51390) 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11e7400640 1 -- 172.21.15.57:0/2341900935 learned_addr learned my addr 172.21.15.57:0/2341900935 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 0x7f11e810c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.018+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8102250 0x7f11e810bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 -- 172.21.15.57:0/2341900935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8102250 msgr2=0x7f11e810bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8102250 0x7f11e810bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 -- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 msgr2=0x7f11e810c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 0x7f11e810c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 -- 172.21.15.57:0/2341900935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e81aab00 con 0x7f11e81018b0 2024-09-12T03:06:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.019+0000 7f11e7400640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.020+0000 7f11dfe00640 1 -- 172.21.15.57:0/2341900935 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800cd80 con 0x7f11e81018b0 2024-09-12T03:06:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.020+0000 7f11dfe00640 1 -- 172.21.15.57:0/2341900935 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d8010910 con 0x7f11e81018b0 2024-09-12T03:06:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.020+0000 7f11dfe00640 1 -- 172.21.15.57:0/2341900935 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8018a00 con 0x7f11e81018b0 2024-09-12T03:06:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.020+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e81aadf0 con 0x7f11e81018b0 2024-09-12T03:06:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.020+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81ab260 con 0x7f11e81018b0 2024-09-12T03:06:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.024+0000 7f11f0e84740 1 Processor -- start 2024-09-12T03:06:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.024+0000 7f11f0e84740 1 -- start start 2024-09-12T03:06:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e203b0 0x55bd64e20790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e2b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bd64d86100 con 0x55bd64e203b0 2024-09-12T03:06:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd64d2e1b0 con 0x55bd64e20cd0 2024-09-12T03:06:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11f0e84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd64d820c0 con 0x55bd64e2b570 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:49102/0 (socket says 172.21.15.57:49102) 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11dfe00640 1 -- 172.21.15.57:0/2341900935 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7f11d8010ac0 con 0x7f11e81018b0 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.025+0000 7f11e6a00640 1 -- 172.21.15.57:0/2303759508 learned_addr learned my addr 172.21.15.57:0/2303759508 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 -- 172.21.15.57:0/2303759508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 msgr2=0x55bd64e2b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e2b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 -- 172.21.15.57:0/2303759508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e203b0 msgr2=0x55bd64e20790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e203b0 0x55bd64e20790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 -- 172.21.15.57:0/2303759508 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd64a99210 con 0x55bd64e2b570 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.026+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009930 tx=0x7f11d4005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90d8d9a0d641b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.027+0000 7f11d3e00640 1 -- 172.21.15.57:0/2303759508 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4035040 con 0x55bd64e2b570 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.027+0000 7f11d3e00640 1 -- 172.21.15.57:0/2303759508 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d4004480 con 0x55bd64e2b570 2024-09-12T03:06:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.027+0000 7f11d3e00640 1 -- 172.21.15.57:0/2303759508 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d40037f0 con 0x55bd64e2b570 2024-09-12T03:06:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 -- 172.21.15.57:0/2303759508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 msgr2=0x55bd64e2d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009930 tx=0x7f11d4005bf0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11dfe00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 0x7f11cc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11dfe00640 1 -- 172.21.15.57:0/2341900935 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f11d80cff30 con 0x7f11e81018b0 2024-09-12T03:06:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 -- 172.21.15.57:0/2303759508 shutdown_connections 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e2b570 0x55bd64e2d960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e2b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2303759508 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e203b0 0x55bd64e20790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11f0e84740 1 -- 172.21.15.57:0/2303759508 >> 172.21.15.57:0/2303759508 conn(0x55bd64e1c920 msgr2=0x55bd64e1cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.028+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 0x7f11cc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.029+0000 7f11f0e84740 1 -- 172.21.15.57:0/2303759508 shutdown_connections 2024-09-12T03:06:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.029+0000 7f11f0e84740 1 -- 172.21.15.57:0/2303759508 wait complete. 2024-09-12T03:06:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.029+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 0x7f11cc0a5360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005fd0 tx=0x7f11e0004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 -- start start 2024-09-12T03:06:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 Processor -- start 2024-09-12T03:06:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e28a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.030+0000 7f11f0e84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bd64e2e140 con 0x55bd64e2b570 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11f0e84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bd64d0f090 con 0x55bd64e20cd0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11f0e84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bd64e2e2c0 con 0x55bd64e203b0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11f0e84740 10 client.0 _ll_register_callbacks cb 0x55bd64d63f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51620/0 (socket says 172.21.15.57:51620) 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e7e00640 1 -- 172.21.15.57:0/2127195692 learned_addr learned my addr 172.21.15.57:0/2127195692 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e7400640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e28a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.031+0000 7f11e7e00640 1 -- 172.21.15.57:0/2127195692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 msgr2=0x55bd64e28550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e28550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7e00640 1 -- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 msgr2=0x55bd64e28a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e28a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7e00640 1 -- 172.21.15.57:0/2127195692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd64b65620 con 0x55bd64e2b570 2024-09-12T03:06:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e6a00640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:06:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7400640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11e7e00640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e0011440 tx=0x7f11e00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67575d8b9449204 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:06:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.032+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e001e830 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d4004480 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f11e001e9d0 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11e002f050 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11f0e84740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd64e4af30 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.033+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d4004480 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11e001c3b0 con 0x55bd64e2b570 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 20 client.15303 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64e4c3f0 0x55bd64e4e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 4 client.15303 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:06:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd64d93830 con 0x55bd64e4c3f0 2024-09-12T03:06:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.034+0000 7f11e7400640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64e4c3f0 0x55bd64e4e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.035+0000 7f11e7400640 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64e4c3f0 0x55bd64e4e7e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82d3e2043e10ef2 server_cookie=758d6cc6d98d0759 in_seq=0 out_seq=0 2024-09-12T03:06:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.035+0000 7f11d0c00640 10 client.15303 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11d0c00640 1 -- 172.21.15.57:0/2127195692 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd64d93830 con 0x55bd64e4c3f0 2024-09-12T03:06:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11d0c00640 10 client.15303 handle_command_reply: tid=0 2024-09-12T03:06:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 client.15303 shutdown 2024-09-12T03:06:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 msgr2=0x55bd64e28fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f11e0011440 tx=0x7f11e00096a0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 shutdown_connections 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64e4c3f0 0x55bd64e4e7e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e2b570 0x55bd64e28fd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e20cd0 0x55bd64e28a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 --2- 172.21.15.57:0/2127195692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.036+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 >> 172.21.15.57:0/2127195692 conn(0x55bd64e1c920 msgr2=0x55bd64e22680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.037+0000 7f11d0c00640 0 client.15303 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.037+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 shutdown_connections 2024-09-12T03:06:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.037+0000 7f11f0e84740 1 -- 172.21.15.57:0/2127195692 wait complete. 2024-09-12T03:06:53.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.037+0000 7f11f0e84740 20 client.15303 trim_cache size 0 max 16384 2024-09-12T03:06:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.067+0000 7f11f0e84740 1 Processor -- start 2024-09-12T03:06:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.067+0000 7f11f0e84740 1 -- start start 2024-09-12T03:06:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.067+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.067+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64e29ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11f0e84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81af570 con 0x55bd64e20cd0 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11f0e84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81ace40 con 0x55bd64e2b570 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11f0e84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11e811c3c0 con 0x55bd64e203b0 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51622/0 (socket says 172.21.15.57:51622) 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 -- 172.21.15.57:0/4206342147 learned_addr learned my addr 172.21.15.57:0/4206342147 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e7400640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e7e00640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64e29ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 -- 172.21.15.57:0/4206342147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 msgr2=0x55bd64e540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 -- 172.21.15.57:0/4206342147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 msgr2=0x55bd64e29ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64e29ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e6a00640 1 -- 172.21.15.57:0/4206342147 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd64e16c20 con 0x55bd64e20cd0 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.068+0000 7f11e7e00640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64e29ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:53.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.069+0000 7f11e6a00640 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f11d4031550 tx=0x7f11d4004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eefc579635d057a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.069+0000 7f11d3400640 1 -- 172.21.15.57:0/4206342147 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4053070 con 0x55bd64e20cd0 2024-09-12T03:06:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.069+0000 7f11d3400640 1 -- 172.21.15.57:0/4206342147 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d40048c0 con 0x55bd64e20cd0 2024-09-12T03:06:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.069+0000 7f11d3400640 1 -- 172.21.15.57:0/4206342147 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4035040 con 0x55bd64e20cd0 2024-09-12T03:06:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 -- 172.21.15.57:0/4206342147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 msgr2=0x55bd64e54630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f11d4031550 tx=0x7f11d4004bf0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 -- 172.21.15.57:0/4206342147 shutdown_connections 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64e29ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e54630 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 --2- 172.21.15.57:0/4206342147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.070+0000 7f11f0e84740 1 -- 172.21.15.57:0/4206342147 >> 172.21.15.57:0/4206342147 conn(0x55bd64e1c920 msgr2=0x55bd64e1cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.071+0000 7f11f0e84740 1 -- 172.21.15.57:0/4206342147 shutdown_connections 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.071+0000 7f11f0e84740 1 -- 172.21.15.57:0/4206342147 wait complete. 2024-09-12T03:06:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.071+0000 7f11f0e84740 1 -- start start 2024-09-12T03:06:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.071+0000 7f11f0e84740 1 Processor -- start 2024-09-12T03:06:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e2ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64eb50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11e811ca30 con 0x55bd64e20cd0 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81ac770 con 0x55bd64e2b570 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81ab9d0 con 0x55bd64e203b0 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11f0e84740 10 client.0 _ll_register_callbacks cb 0x7f11e0003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.072+0000 7f11e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:49130/0 (socket says 172.21.15.57:49130) 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 -- 172.21.15.57:0/293117890 learned_addr learned my addr 172.21.15.57:0/293117890 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e6a00640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e2ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 -- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 msgr2=0x55bd64eb50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64eb50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 -- 172.21.15.57:0/293117890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 msgr2=0x55bd64e2ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e2ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 -- 172.21.15.57:0/293117890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd64eb55f0 con 0x55bd64e203b0 2024-09-12T03:06:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e6a00640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e2ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:06:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11e7400640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11d80bc0d0 tx=0x7f11d8010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a9c92703047bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.073+0000 7f11d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80a4ad0 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f11d80984d0 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d40048c0 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11d80a4640 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11f0e84740 10 client.25360 fetch_fsmap learned FSMap version 26 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd64eb6710 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.074+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11d40048c0 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.075+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11d809b2b0 con 0x55bd64e203b0 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 10 client.25360 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 10 client.25360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 20 client.25360 populate_metadata read hostname 'smithi057' 2024-09-12T03:06:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64eb6e90 0x55bd64eb92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:06:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 4 client.25360 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:06:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd64d8db80 con 0x55bd64eb6e90 2024-09-12T03:06:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.076+0000 7f11e6a00640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64eb6e90 0x55bd64eb92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:06:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.077+0000 7f11e6a00640 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64eb6e90 0x55bd64eb92d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0c56b82bf34252 server_cookie=16d5d9538806e5b6 in_seq=0 out_seq=0 2024-09-12T03:06:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.077+0000 7f11d0c00640 10 client.25360 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.080+0000 7f11d0c00640 1 -- 172.21.15.57:0/293117890 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd64d8db80 con 0x55bd64eb6e90 2024-09-12T03:06:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.080+0000 7f11d0c00640 10 client.25360 handle_command_reply: tid=0 2024-09-12T03:06:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.080+0000 7f11f0e84740 1 client.25360 shutdown 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 msgr2=0x55bd64e2e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f11d80bc0d0 tx=0x7f11d8010f90 comp rx=0 tx=0).stop 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 shutdown_connections 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bd64eb6e90 0x55bd64eb92d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bd64e2b570 0x55bd64eb50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bd64e20cd0 0x55bd64e2ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 --2- 172.21.15.57:0/293117890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd64e203b0 0x55bd64e2e720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11d0c00640 0 client.25360 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:06:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.081+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 >> 172.21.15.57:0/293117890 conn(0x55bd64e1c920 msgr2=0x55bd64e21e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.082+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 shutdown_connections 2024-09-12T03:06:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.082+0000 7f11f0e84740 1 -- 172.21.15.57:0/293117890 wait complete. 2024-09-12T03:06:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.082+0000 7f11f0e84740 20 client.25360 trim_cache size 0 max 16384 2024-09-12T03:06:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:06:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:06:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:06:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:06:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 msgr2=0x7f11cc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 0x7f11cc0a5360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11e0005fd0 tx=0x7f11e0004a40 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 msgr2=0x7f11e810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e9f0 tx=0x7f11d800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 shutdown_connections 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11cc0a2ea0 0x7f11cc0a5360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11e81078c0 0x7f11e810c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11e8102250 0x7f11e810bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 --2- 172.21.15.57:0/2341900935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e81018b0 0x7f11e810b9a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.083+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 >> 172.21.15.57:0/2341900935 conn(0x7f11e80fd5d0 msgr2=0x7f11e80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.084+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 shutdown_connections 2024-09-12T03:06:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:06:53.084+0000 7f11eca00640 1 -- 172.21.15.57:0/2341900935 wait complete. 2024-09-12T03:06:53.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:53 smithi057 ceph-mon[30361]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:06:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:53 smithi204 ceph-mon[35013]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:06:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:53 smithi181 ceph-mon[35320]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-09-12T03:06:55.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:55 smithi057 ceph-mon[30361]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:06:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:55 smithi204 ceph-mon[35013]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:06:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:55 smithi181 ceph-mon[35320]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T03:06:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:57 smithi057 ceph-mon[30361]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:57 smithi204 ceph-mon[35013]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:57 smithi181 ceph-mon[35320]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:06:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:06:59 smithi057 ceph-mon[30361]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:06:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:06:59 smithi204 ceph-mon[35013]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:06:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:06:59 smithi181 ceph-mon[35320]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T03:07:01.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:01 smithi057 ceph-mon[30361]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:07:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:01 smithi204 ceph-mon[35013]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:07:01.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:01 smithi181 ceph-mon[35320]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T03:07:02.138 INFO:tasks.workunit.client.0.smithi057.stderr:Updating files: 62% (8040/12903) Updating files: 63% (8129/12903) Updating files: 64% (8258/12903) Updating files: 65% (8387/12903) Updating files: 66% (8516/12903) Updating files: 67% (8646/12903) Updating files: 68% (8775/12903) Updating files: 69% (8904/12903) Updating files: 70% (9033/12903) Updating files: 71% (9162/12903) Updating files: 72% (9291/12903) Updating files: 73% (9420/12903) Updating files: 74% (9549/12903) Updating files: 75% (9678/12903) Updating files: 76% (9807/12903) Updating files: 77% (9936/12903) Updating files: 78% (10065/12903) Updating files: 79% (10194/12903) Updating files: 80% (10323/12903) Updating files: 81% (10452/12903) Updating files: 82% (10581/12903) Updating files: 83% (10710/12903) Updating files: 84% (10839/12903) Updating files: 85% (10968/12903) Updating files: 86% (11097/12903) Updating files: 87% (11226/12903) Updating files: 88% (11355/12903) Updating files: 89% (11484/12903) Updating files: 90% (11613/12903) Updating files: 91% (11742/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 98% (12696/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:07:02.831 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: git switch -c 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr:Or undo this operation with: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: git switch - 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:02.832 INFO:tasks.workunit.client.0.smithi057.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:07:02.847 DEBUG:teuthology.orchestra.run.smithi057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:07:02.938 INFO:tasks.workunit.client.0.smithi057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:07:02.943 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:07:02.943 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:07:03.624 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:07:03.747 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:07:03.817 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:07:03.822 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:07:03.822 INFO:tasks.workunit.client.0.smithi057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:07:03.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:03 smithi057 ceph-mon[30361]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-12T03:07:03.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:03 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:03 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:03.889 INFO:tasks.workunit.client.0.smithi057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:07:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:03 smithi204 ceph-mon[35013]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-12T03:07:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:03 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:03 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:03.896 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:07:03.896 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:07:03.932 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-12T03:07:03.933 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-12T03:07:03.934 DEBUG:teuthology.orchestra.run.smithi057:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-12T03:07:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:03 smithi181 ceph-mon[35320]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-12T03:07:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:03 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:07:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:03 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:04.022 INFO:tasks.workunit.client.0.smithi057.stdout:getting iogen 2024-09-12T03:07:04.023 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'getting iogen' 2024-09-12T03:07:04.023 INFO:tasks.workunit.client.0.smithi057.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-12T03:07:04.057 INFO:tasks.workunit.client.0.smithi057.stderr:--2024-09-12 03:07:04-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-12T03:07:04.058 INFO:tasks.workunit.client.0.smithi057.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-12T03:07:04.080 INFO:tasks.workunit.client.0.smithi057.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-12T03:07:04.100 INFO:tasks.workunit.client.0.smithi057.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-12T03:07:04.101 INFO:tasks.workunit.client.0.smithi057.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-12T03:07:04.102 INFO:tasks.workunit.client.0.smithi057.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr: 0K ....... 100% 720M=0s 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr:2024-09-12 03:07:04 (720 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr: 2024-09-12T03:07:04.107 INFO:tasks.workunit.client.0.smithi057.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-12T03:07:04.125 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0 2024-09-12T03:07:04.127 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/src 2024-09-12T03:07:04.128 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/src/iogen.8 2024-09-12T03:07:04.129 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/src/iogen.c 2024-09-12T03:07:04.131 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/README 2024-09-12T03:07:04.132 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/Makefile.bsd 2024-09-12T03:07:04.133 INFO:tasks.workunit.client.0.smithi057.stdout:iogen_3.1p0/Makefile 2024-09-12T03:07:04.135 INFO:tasks.workunit.client.0.smithi057.stderr:+ cd iogen_3.1p0 2024-09-12T03:07:04.135 INFO:tasks.workunit.client.0.smithi057.stdout:making iogen 2024-09-12T03:07:04.135 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'making iogen' 2024-09-12T03:07:04.135 INFO:tasks.workunit.client.0.smithi057.stderr:+ make 2024-09-12T03:07:04.144 INFO:tasks.workunit.client.0.smithi057.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-12T03:07:04.411 INFO:tasks.workunit.client.0.smithi057.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-12T03:07:04.527 INFO:tasks.workunit.client.0.smithi057.stdout:running iogen 2024-09-12T03:07:04.527 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'running iogen' 2024-09-12T03:07:04.527 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./iogen -n 5 -s 2g 2024-09-12T03:07:04.529 INFO:tasks.workunit.client.0.smithi057.stderr:child spawned 86731 2024-09-12T03:07:04.529 INFO:tasks.workunit.client.0.smithi057.stderr:child spawned 86732 2024-09-12T03:07:04.529 INFO:tasks.workunit.client.0.smithi057.stderr:child spawned 86733 2024-09-12T03:07:04.529 INFO:tasks.workunit.client.0.smithi057.stderr:child spawned 86734 2024-09-12T03:07:04.530 INFO:tasks.workunit.client.0.smithi057.stderr:child spawned 86735 2024-09-12T03:07:04.530 INFO:tasks.workunit.client.0.smithi057.stdout:sleep for 10 min 2024-09-12T03:07:04.530 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'sleep for 10 min' 2024-09-12T03:07:04.530 INFO:tasks.workunit.client.0.smithi057.stderr:+ sleep 600 2024-09-12T03:07:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:04 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:04 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:04.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:04 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:04.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:04 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:04.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:04 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:04.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:04 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:04.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:04 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:07:04.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:04 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:07:04.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:04 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:07:05.877 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:05 smithi057 ceph-mon[30361]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:05 smithi204 ceph-mon[35013]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:05.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:05 smithi181 ceph-mon[35320]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.443+0000 7f9ef5200640 1 -- 172.21.15.57:0/3353625619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef0108290 msgr2=0x7f9ef0108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.443+0000 7f9ef5200640 1 --2- 172.21.15.57:0/3353625619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef0108290 0x7f9ef0108670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009ca0 tx=0x7f9ee802a8f0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.446+0000 7f9ef5200640 1 -- 172.21.15.57:0/3353625619 shutdown_connections 2024-09-12T03:07:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.446+0000 7f9ef5200640 1 --2- 172.21.15.57:0/3353625619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef010d2d0 0x7f9ef0114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.446+0000 7f9ef5200640 1 --2- 172.21.15.57:0/3353625619 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef0108c40 0x7f9ef010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.446+0000 7f9ef5200640 1 --2- 172.21.15.57:0/3353625619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef0108290 0x7f9ef0108670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.446+0000 7f9ef5200640 1 -- 172.21.15.57:0/3353625619 >> 172.21.15.57:0/3353625619 conn(0x7f9ef00778f0 msgr2=0x7f9ef0079d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.450+0000 7f9ef5200640 1 -- 172.21.15.57:0/3353625619 shutdown_connections 2024-09-12T03:07:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.450+0000 7f9ef5200640 1 -- 172.21.15.57:0/3353625619 wait complete. 2024-09-12T03:07:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.451+0000 7f9ef5200640 1 Processor -- start 2024-09-12T03:07:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.462+0000 7f9ef5200640 1 -- start start 2024-09-12T03:07:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.463+0000 7f9ef5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 0x7f9ef01a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.463+0000 7f9ef5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.463+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 0x7f9ef01a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.463+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 0x7f9ef01a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35686/0 (socket says 172.21.15.57:35686) 2024-09-12T03:07:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.468+0000 7f9eeea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.468+0000 7f9eeea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50326/0 (socket says 172.21.15.57:50326) 2024-09-12T03:07:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.468+0000 7f9eef400640 1 -- 172.21.15.57:0/116724759 learned_addr learned my addr 172.21.15.57:0/116724759 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.475+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef019e6c0 0x7f9ef019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.477+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0118390 con 0x7f9ef010d2d0 2024-09-12T03:07:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.477+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0118210 con 0x7f9ef019e6c0 2024-09-12T03:07:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.477+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0118510 con 0x7f9ef0108290 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.478+0000 7f9eeea00640 1 -- 172.21.15.57:0/116724759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 msgr2=0x7f9ef01a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.478+0000 7f9eeea00640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 0x7f9ef01a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.478+0000 7f9eeea00640 1 -- 172.21.15.57:0/116724759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef019e6c0 msgr2=0x7f9ef019eb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.478+0000 7f9eeea00640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef019e6c0 0x7f9ef019eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.478+0000 7f9eeea00640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef019f2c0 con 0x7f9ef010d2d0 2024-09-12T03:07:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.479+0000 7f9eeea00640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000ed30 tx=0x7f9ee000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.480+0000 7f9ee7e00640 1 -- 172.21.15.57:0/116724759 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000cdb0 con 0x7f9ef010d2d0 2024-09-12T03:07:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.481+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01ab3a0 con 0x7f9ef010d2d0 2024-09-12T03:07:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.481+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01ab800 con 0x7f9ef010d2d0 2024-09-12T03:07:06.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.481+0000 7f9ee7e00640 1 -- 172.21.15.57:0/116724759 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0009370 con 0x7f9ef010d2d0 2024-09-12T03:07:06.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.482+0000 7f9ee7e00640 1 -- 172.21.15.57:0/116724759 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000cdb0 con 0x7f9ef010d2d0 2024-09-12T03:07:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.490+0000 7f9ee7e00640 1 -- 172.21.15.57:0/116724759 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7f9ee0003ce0 con 0x7f9ef010d2d0 2024-09-12T03:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.492+0000 7f9ee7e00640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 0x7f9ed40a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.494+0000 7f9eef400640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 0x7f9ed40a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.494+0000 7f9ee7e00640 1 -- 172.21.15.57:0/116724759 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9ee0009e00 con 0x7f9ef010d2d0 2024-09-12T03:07:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.496+0000 7f9eef400640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 0x7f9ed40a5760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009430 tx=0x7f9ee8009360 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.551+0000 7f9ef983f740 1 Processor -- start 2024-09-12T03:07:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.552+0000 7f9ef983f740 1 -- start start 2024-09-12T03:07:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.556+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc12e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.556+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc139030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.556+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.558+0000 7f9eefe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.558+0000 7f9eefe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:34390/0 (socket says 172.21.15.57:34390) 2024-09-12T03:07:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.561+0000 7f9eefe00640 1 -- 172.21.15.57:0/1212724577 learned_addr learned my addr 172.21.15.57:0/1212724577 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.561+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558cfc094100 con 0x558cfc12ecd0 2024-09-12T03:07:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.561+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558cfc03c1b0 con 0x558cfc139570 2024-09-12T03:07:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.561+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558cfc0900c0 con 0x558cfc12e3b0 2024-09-12T03:07:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.562+0000 7f9eefe00640 1 -- 172.21.15.57:0/1212724577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 msgr2=0x558cfc12e790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eeea00640 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eefe00640 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc12e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eefe00640 1 -- 172.21.15.57:0/1212724577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 msgr2=0x558cfc139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eefe00640 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eefe00640 1 -- 172.21.15.57:0/1212724577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfbda7210 con 0x558cfc139570 2024-09-12T03:07:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.565+0000 7f9eeea00640 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.567+0000 7f9eefe00640 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0096e0 tx=0x7f9edc009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149fa1c1a5eb1d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.567+0000 7f9ee4c00640 1 -- 172.21.15.57:0/1212724577 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc015020 con 0x558cfc139570 2024-09-12T03:07:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.568+0000 7f9ee4c00640 1 -- 172.21.15.57:0/1212724577 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc00bda0 con 0x558cfc139570 2024-09-12T03:07:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.568+0000 7f9ee4c00640 1 -- 172.21.15.57:0/1212724577 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc015020 con 0x558cfc139570 2024-09-12T03:07:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.569+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 msgr2=0x558cfc13b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.569+0000 7f9ef983f740 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0096e0 tx=0x7f9edc009ba0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.569+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 shutdown_connections 2024-09-12T03:07:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.569+0000 7f9ef983f740 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc13b960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.570+0000 7f9ef983f740 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.570+0000 7f9ef983f740 1 --2- 172.21.15.57:0/1212724577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc12e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.570+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 >> 172.21.15.57:0/1212724577 conn(0x558cfc12a920 msgr2=0x558cfc12ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.570+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 shutdown_connections 2024-09-12T03:07:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.570+0000 7f9ef983f740 1 -- 172.21.15.57:0/1212724577 wait complete. 2024-09-12T03:07:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.572+0000 7f9ef983f740 1 -- start start 2024-09-12T03:07:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.572+0000 7f9ef983f740 1 Processor -- start 2024-09-12T03:07:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.575+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.576+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc1369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.576+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc136f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.576+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.576+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35704/0 (socket says 172.21.15.57:35704) 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.576+0000 7f9eef400640 1 -- 172.21.15.57:0/390556761 learned_addr learned my addr 172.21.15.57:0/390556761 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.577+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558cfc13c050 con 0x558cfc12ecd0 2024-09-12T03:07:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.577+0000 7f9eefe00640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc136f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.577+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558cfc01d090 con 0x558cfc139570 2024-09-12T03:07:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.577+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558cfc13c1d0 con 0x558cfc12e3b0 2024-09-12T03:07:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.578+0000 7f9eefe00640 1 -- 172.21.15.57:0/390556761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 msgr2=0x558cfc1364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.578+0000 7f9eefe00640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.578+0000 7f9eefe00640 1 -- 172.21.15.57:0/390556761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 msgr2=0x558cfc1369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.578+0000 7f9eefe00640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc1369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.579+0000 7f9ef983f740 10 client.0 _ll_register_callbacks cb 0x558cfc071f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.579+0000 7f9eefe00640 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfbe73620 con 0x558cfc139570 2024-09-12T03:07:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.579+0000 7f9eef400640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.580+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9edc00bda0 con 0x558cfc139570 2024-09-12T03:07:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.580+0000 7f9eefe00640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc136f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x558cfc130f50 tx=0x7f9edc011650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938599a982428a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.581+0000 7f9ed9600640 10 client.25355 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:07:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.581+0000 7f9ed9600640 1 -- 172.21.15.57:0/390556761 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011e60 con 0x558cfc139570 2024-09-12T03:07:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.581+0000 7f9ed9600640 1 -- 172.21.15.57:0/390556761 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9edc00b8c0 con 0x558cfc139570 2024-09-12T03:07:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.582+0000 7f9ed9600640 1 -- 172.21.15.57:0/390556761 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9edc02d050 con 0x558cfc139570 2024-09-12T03:07:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.582+0000 7f9ef983f740 10 client.25355 fetch_fsmap learned FSMap version 26 2024-09-12T03:07:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.582+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=26}) -- 0x558cfc147c70 con 0x558cfc139570 2024-09-12T03:07:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.584+0000 7f9ed9600640 1 -- 172.21.15.57:0/390556761 <== mon.2 v2:172.21.15.181:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9edc004040 con 0x558cfc139570 2024-09-12T03:07:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 10 client.25355 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:07:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 20 client.25355 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc149510 0x558cfc14b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 4 client.25355 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9edc00bda0 con 0x558cfc149510 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.585+0000 7f9eef400640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc149510 0x558cfc14b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.586+0000 7f9ed9600640 10 client.25355 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.586+0000 7f9eef400640 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc149510 0x558cfc14b9d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516f0e69de7bc2c5 server_cookie=b596cfc029ea569a in_seq=0 out_seq=0 2024-09-12T03:07:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.587+0000 7f9ed9600640 1 -- 172.21.15.57:0/390556761 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9edc00bda0 con 0x558cfc149510 2024-09-12T03:07:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.587+0000 7f9ed9600640 10 client.25355 handle_command_reply: tid=0 2024-09-12T03:07:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.587+0000 7f9ef983f740 1 client.25355 shutdown 2024-09-12T03:07:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 msgr2=0x558cfc136f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc136f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x558cfc130f50 tx=0x7f9edc011650 comp rx=0 tx=0).stop 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 shutdown_connections 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc149510 0x558cfc14b9d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc136f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc1369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 --2- 172.21.15.57:0/390556761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc1364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 >> 172.21.15.57:0/390556761 conn(0x558cfc12a920 msgr2=0x558cfc130680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.588+0000 7f9ed9600640 0 client.25355 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.589+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 shutdown_connections 2024-09-12T03:07:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.589+0000 7f9ef983f740 1 -- 172.21.15.57:0/390556761 wait complete. 2024-09-12T03:07:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.589+0000 7f9ef983f740 20 client.25355 trim_cache size 0 max 16384 2024-09-12T03:07:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.648+0000 7f9ef983f740 1 Processor -- start 2024-09-12T03:07:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.648+0000 7f9ef983f740 1 -- start start 2024-09-12T03:07:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.649+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12e3b0 0x558cfc1369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.649+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.650+0000 7f9eef400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12e3b0 0x558cfc1369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.650+0000 7f9eeea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.650+0000 7f9eeea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:34414/0 (socket says 172.21.15.57:34414) 2024-09-12T03:07:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9eeea00640 1 -- 172.21.15.57:0/3071772833 learned_addr learned my addr 172.21.15.57:0/3071772833 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 0x558cfc1336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0012870 con 0x558cfc12e3b0 2024-09-12T03:07:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0010140 con 0x558cfc12ecd0 2024-09-12T03:07:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed000df90 con 0x558cfc139570 2024-09-12T03:07:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.651+0000 7f9eefe00640 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 0x558cfc1336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.654+0000 7f9eeea00640 1 -- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 msgr2=0x558cfc1336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.654+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 0x558cfc1336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.654+0000 7f9eeea00640 1 -- 172.21.15.57:0/3071772833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12e3b0 msgr2=0x558cfc1369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.654+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12e3b0 0x558cfc1369e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.654+0000 7f9eeea00640 1 -- 172.21.15.57:0/3071772833 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cfc12a750 con 0x558cfc12ecd0 2024-09-12T03:07:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.656+0000 7f9eefe00640 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 0x558cfc1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.657+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00a36e0 tx=0x7f9ee00a2660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551c653321ffb7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.657+0000 7f9edbe00640 1 -- 172.21.15.57:0/3071772833 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0097a60 con 0x558cfc12ecd0 2024-09-12T03:07:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.658+0000 7f9edbe00640 1 -- 172.21.15.57:0/3071772833 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee00056a0 con 0x558cfc12ecd0 2024-09-12T03:07:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.658+0000 7f9edbe00640 1 -- 172.21.15.57:0/3071772833 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0097c00 con 0x558cfc12ecd0 2024-09-12T03:07:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.662+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 msgr2=0x558cfc136f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.663+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00a36e0 tx=0x7f9ee00a2660 comp rx=0 tx=0).stop 2024-09-12T03:07:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.668+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 shutdown_connections 2024-09-12T03:07:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.668+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc139570 0x558cfc1336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.668+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc12ecd0 0x558cfc136f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.668+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3071772833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12e3b0 0x558cfc1369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.668+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 >> 172.21.15.57:0/3071772833 conn(0x558cfc12a920 msgr2=0x558cfc131750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.675+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 shutdown_connections 2024-09-12T03:07:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.675+0000 7f9ef983f740 1 -- 172.21.15.57:0/3071772833 wait complete. 2024-09-12T03:07:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.676+0000 7f9ef983f740 1 -- start start 2024-09-12T03:07:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.681+0000 7f9ef983f740 1 Processor -- start 2024-09-12T03:07:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed000e600 con 0x558cfc12ecd0 2024-09-12T03:07:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed000fa70 con 0x558cfc139570 2024-09-12T03:07:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed000ecd0 con 0x558cfc12e3b0 2024-09-12T03:07:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.687+0000 7f9ef983f740 10 client.0 _ll_register_callbacks cb 0x7f9edc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.690+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.690+0000 7f9eef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35730/0 (socket says 172.21.15.57:35730) 2024-09-12T03:07:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.690+0000 7f9eef400640 1 -- 172.21.15.57:0/3506543967 learned_addr learned my addr 172.21.15.57:0/3506543967 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.695+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.695+0000 7f9eefe00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.696+0000 7f9eef400640 1 -- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 msgr2=0x558cfc14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.696+0000 7f9eef400640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.696+0000 7f9eef400640 1 -- 172.21.15.57:0/3506543967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 msgr2=0x558cfc14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.698+0000 7f9eef400640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.698+0000 7f9eef400640 1 -- 172.21.15.57:0/3506543967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cfc13c2c0 con 0x558cfc12e3b0 2024-09-12T03:07:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.699+0000 7f9eefe00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.702+0000 7f9eef400640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x558cfc03c1b0 tx=0x7f9ee803e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638beed8e1ad826f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.702+0000 7f9ed9600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:07:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.705+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.712+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80307d0 con 0x558cfc12e3b0 2024-09-12T03:07:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.712+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9ee80317f0 con 0x558cfc12e3b0 2024-09-12T03:07:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.713+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee00056a0 con 0x558cfc12e3b0 2024-09-12T03:07:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.716+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ee8052410 con 0x558cfc12e3b0 2024-09-12T03:07:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.721+0000 7f9ef983f740 10 client.25372 fetch_fsmap learned FSMap version 26 2024-09-12T03:07:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.721+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cfc1c4530 con 0x558cfc12e3b0 2024-09-12T03:07:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.721+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee00056a0 con 0x558cfc12e3b0 2024-09-12T03:07:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.723+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ee8043040 con 0x558cfc12e3b0 2024-09-12T03:07:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.723+0000 7f9ef983f740 10 client.25372 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T03:07:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.723+0000 7f9ef983f740 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.723+0000 7f9ef983f740 20 client.25372 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.724+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc1c4c00 0x558cfc1c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.724+0000 7f9ef983f740 4 client.25372 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:07:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.724+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558cfc0a1830 con 0x558cfc1c4c00 2024-09-12T03:07:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.724+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc1c4c00 0x558cfc1c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.725+0000 7f9ed9600640 10 client.25372 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.725+0000 7f9eeea00640 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc1c4c00 0x558cfc1c6ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3243d35abcc3602a server_cookie=ff1b5230b8f8acfa in_seq=0 out_seq=0 2024-09-12T03:07:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.787+0000 7f9ed9600640 1 -- 172.21.15.57:0/3506543967 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558cfc0a1830 con 0x558cfc1c4c00 2024-09-12T03:07:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.787+0000 7f9ed9600640 10 client.25372 handle_command_reply: tid=0 2024-09-12T03:07:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.787+0000 7f9ef983f740 1 client.25372 shutdown 2024-09-12T03:07:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.787+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 msgr2=0x558cfc14e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.787+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x558cfc03c1b0 tx=0x7f9ee803e040 comp rx=0 tx=0).stop 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 shutdown_connections 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558cfc1c4c00 0x558cfc1c6ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558cfc139570 0x558cfc14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558cfc12ecd0 0x558cfc14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 --2- 172.21.15.57:0/3506543967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558cfc12e3b0 0x558cfc14e170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.790+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 >> 172.21.15.57:0/3506543967 conn(0x558cfc12a920 msgr2=0x558cfc1445f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.791+0000 7f9ed9600640 0 client.25372 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.791+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 shutdown_connections 2024-09-12T03:07:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.792+0000 7f9ef983f740 1 -- 172.21.15.57:0/3506543967 wait complete. 2024-09-12T03:07:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.792+0000 7f9ef983f740 20 client.25372 trim_cache size 0 max 16384 2024-09-12T03:07:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:07:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:07:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:07:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:07:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.793+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 msgr2=0x7f9ed40a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.794+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 0x7f9ed40a5760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009430 tx=0x7f9ee8009360 comp rx=0 tx=0).stop 2024-09-12T03:07:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.794+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 msgr2=0x7f9ef01a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.794+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000ed30 tx=0x7f9ee000c6a0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 shutdown_connections 2024-09-12T03:07:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ef019e6c0 0x7f9ef019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9ed40a32a0 0x7f9ed40a5760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9ef010d2d0 0x7f9ef01a4b40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 --2- 172.21.15.57:0/116724759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ef0108290 0x7f9ef01a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:06.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.795+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 >> 172.21.15.57:0/116724759 conn(0x7f9ef00778f0 msgr2=0x7f9ef0078c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.799+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 shutdown_connections 2024-09-12T03:07:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:06.799+0000 7f9ef5200640 1 -- 172.21.15.57:0/116724759 wait complete. 2024-09-12T03:07:07.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:07 smithi057 ceph-mon[30361]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:07 smithi204 ceph-mon[35013]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:07 smithi181 ceph-mon[35320]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-12T03:07:08.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:08 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:08 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:08 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:09.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:09 smithi057 ceph-mon[30361]: pgmap v39: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 21 MiB/s wr, 36 op/s 2024-09-12T03:07:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:09 smithi204 ceph-mon[35013]: pgmap v39: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 21 MiB/s wr, 36 op/s 2024-09-12T03:07:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:09 smithi181 ceph-mon[35320]: pgmap v39: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 21 MiB/s wr, 36 op/s 2024-09-12T03:07:10.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:10 smithi057 ceph-mon[30361]: pgmap v40: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-09-12T03:07:10.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:10 smithi204 ceph-mon[35013]: pgmap v40: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-09-12T03:07:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:10 smithi181 ceph-mon[35320]: pgmap v40: 129 pgs: 129 active+clean; 250 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 21 MiB/s wr, 33 op/s 2024-09-12T03:07:13.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:12 smithi057 ceph-mon[30361]: pgmap v41: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 125 MiB/s wr, 71 op/s 2024-09-12T03:07:13.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:12 smithi204 ceph-mon[35013]: pgmap v41: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 125 MiB/s wr, 71 op/s 2024-09-12T03:07:13.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:12 smithi181 ceph-mon[35320]: pgmap v41: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 125 MiB/s wr, 71 op/s 2024-09-12T03:07:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:13 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:13 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:13 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:07:15.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:14 smithi204 ceph-mon[35013]: pgmap v42: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:15.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:14 smithi181 ceph-mon[35320]: pgmap v42: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:15.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:14 smithi057 ceph-mon[30361]: pgmap v42: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:17.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:16 smithi204 ceph-mon[35013]: pgmap v43: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:17.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:16 smithi181 ceph-mon[35320]: pgmap v43: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:17.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:16 smithi057 ceph-mon[30361]: pgmap v43: 129 pgs: 129 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 137 MiB/s wr, 57 op/s 2024-09-12T03:07:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:18 smithi204 ceph-mon[35013]: pgmap v44: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 254 MiB/s wr, 90 op/s 2024-09-12T03:07:19.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:18 smithi181 ceph-mon[35320]: pgmap v44: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 254 MiB/s wr, 90 op/s 2024-09-12T03:07:19.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:18 smithi057 ceph-mon[30361]: pgmap v44: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 254 MiB/s wr, 90 op/s 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.078+0000 7f5856600640 1 -- 172.21.15.57:0/282964035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074590 msgr2=0x7f5850074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.078+0000 7f5856600640 1 --2- 172.21.15.57:0/282964035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074590 0x7f5850074970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f584002a480 comp rx=0 tx=0).stop 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.079+0000 7f5856600640 1 -- 172.21.15.57:0/282964035 shutdown_connections 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.079+0000 7f5856600640 1 --2- 172.21.15.57:0/282964035 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f5850114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.079+0000 7f5856600640 1 --2- 172.21.15.57:0/282964035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5850074eb0 0x7f58501125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.079+0000 7f5856600640 1 --2- 172.21.15.57:0/282964035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074590 0x7f5850074970 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5840009900 tx=0x7f584002a480 comp rx=0 tx=0).stop 2024-09-12T03:07:20.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.079+0000 7f5856600640 1 -- 172.21.15.57:0/282964035 >> 172.21.15.57:0/282964035 conn(0x7f58500fd7d0 msgr2=0x7f58500ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.080+0000 7f5856600640 1 -- 172.21.15.57:0/282964035 shutdown_connections 2024-09-12T03:07:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.080+0000 7f5856600640 1 -- 172.21.15.57:0/282964035 wait complete. 2024-09-12T03:07:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.088+0000 7f5856600640 1 Processor -- start 2024-09-12T03:07:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.088+0000 7f5856600640 1 -- start start 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 0x7f58501a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f58501a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f58501184f0 con 0x7f5850112ae0 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5850118370 con 0x7f5850074eb0 2024-09-12T03:07:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f5856600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5850118670 con 0x7f585019e9a0 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f584fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f584fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:47978/0 (socket says 172.21.15.57:47978) 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.089+0000 7f584fe00640 1 -- 172.21.15.57:0/2175212969 learned_addr learned my addr 172.21.15.57:0/2175212969 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.090+0000 7f584fe00640 1 -- 172.21.15.57:0/2175212969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 msgr2=0x7f58501a47f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.090+0000 7f584f400640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 0x7f58501a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.090+0000 7f584fe00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 0x7f58501a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.091+0000 7f584ea00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f58501a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.091+0000 7f584fe00640 1 -- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 msgr2=0x7f58501a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.091+0000 7f584fe00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f58501a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.091+0000 7f584fe00640 1 -- 172.21.15.57:0/2175212969 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585019f4c0 con 0x7f585019e9a0 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.092+0000 7f584fe00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f584800eaa0 tx=0x7f584800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.092+0000 7f5847e00640 1 -- 172.21.15.57:0/2175212969 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584800cdb0 con 0x7f585019e9a0 2024-09-12T03:07:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.092+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585019f7b0 con 0x7f585019e9a0 2024-09-12T03:07:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.093+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58501aba50 con 0x7f585019e9a0 2024-09-12T03:07:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.093+0000 7f584f400640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 0x7f58501a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.093+0000 7f584ea00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f58501a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.094+0000 7f5847e00640 1 -- 172.21.15.57:0/2175212969 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5848005020 con 0x7f585019e9a0 2024-09-12T03:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.095+0000 7f5847e00640 1 -- 172.21.15.57:0/2175212969 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584800cdb0 con 0x7f585019e9a0 2024-09-12T03:07:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.106+0000 7f5847e00640 1 -- 172.21.15.57:0/2175212969 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 26) ==== 138117+0+0 (secure 0 0 0) 0x7f5848004a40 con 0x7f585019e9a0 2024-09-12T03:07:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.109+0000 7f58595ca740 1 Processor -- start 2024-09-12T03:07:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 -- start start 2024-09-12T03:07:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4e3a0 0x5599e4a50790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5599e49a9100 con 0x5599e4a43b00 2024-09-12T03:07:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5599e49511b0 con 0x5599e4a431e0 2024-09-12T03:07:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.111+0000 7f58595ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5599e49a50c0 con 0x5599e4a4e3a0 2024-09-12T03:07:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.113+0000 7f5847e00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 0x7f58340a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.113+0000 7f584fe00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 0x7f58340a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.113+0000 7f584ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.113+0000 7f584ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60566/0 (socket says 172.21.15.57:60566) 2024-09-12T03:07:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.114+0000 7f584ea00640 1 -- 172.21.15.57:0/641067365 learned_addr learned my addr 172.21.15.57:0/641067365 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.114+0000 7f584ea00640 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4e3a0 0x5599e4a50790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.114+0000 7f584ea00640 1 -- 172.21.15.57:0/641067365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4e3a0 msgr2=0x5599e4a50790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.114+0000 7f584ea00640 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4e3a0 0x5599e4a50790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.114+0000 7f584ea00640 1 -- 172.21.15.57:0/641067365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 msgr2=0x5599e4a4de60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:07:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.115+0000 7f5847e00640 1 -- 172.21.15.57:0/2175212969 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f584800cbb0 con 0x7f585019e9a0 2024-09-12T03:07:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.116+0000 7f584fe00640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 0x7f58340a58e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f58480ccb50 tx=0x7f58480d5240 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.117+0000 7f584ea00640 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.117+0000 7f584ea00640 1 -- 172.21.15.57:0/641067365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e46bc210 con 0x5599e4a431e0 2024-09-12T03:07:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.119+0000 7f584ea00640 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f583c0134e0 tx=0x7f583c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ad287f8dc710c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.122+0000 7f5844c00640 1 -- 172.21.15.57:0/641067365 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c019070 con 0x5599e4a431e0 2024-09-12T03:07:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.122+0000 7f5844c00640 1 -- 172.21.15.57:0/641067365 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c004230 con 0x5599e4a431e0 2024-09-12T03:07:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.122+0000 7f5844c00640 1 -- 172.21.15.57:0/641067365 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c011700 con 0x5599e4a431e0 2024-09-12T03:07:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.125+0000 7f58595ca740 1 -- 172.21.15.57:0/641067365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 msgr2=0x5599e4a435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.125+0000 7f58595ca740 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f583c0134e0 tx=0x7f583c0096e0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.127+0000 7f58595ca740 1 -- 172.21.15.57:0/641067365 shutdown_connections 2024-09-12T03:07:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.127+0000 7f58595ca740 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4e3a0 0x5599e4a50790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.127+0000 7f58595ca740 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.127+0000 7f58595ca740 1 --2- 172.21.15.57:0/641067365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a431e0 0x5599e4a435c0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f583c0134e0 tx=0x7f583c0096e0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.127+0000 7f58595ca740 1 -- 172.21.15.57:0/641067365 >> 172.21.15.57:0/641067365 conn(0x5599e4a3f750 msgr2=0x5599e4a3fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.128+0000 7f58595ca740 1 -- 172.21.15.57:0/641067365 shutdown_connections 2024-09-12T03:07:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.128+0000 7f58595ca740 1 -- 172.21.15.57:0/641067365 wait complete. 2024-09-12T03:07:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.129+0000 7f58595ca740 1 -- start start 2024-09-12T03:07:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.129+0000 7f58595ca740 1 Processor -- start 2024-09-12T03:07:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5599e4a61f80 con 0x5599e4a43b00 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5599e4932090 con 0x5599e4a4e3a0 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5599e4a62100 con 0x5599e4a4bec0 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.130+0000 7f58595ca740 10 client.0 _ll_register_callbacks cb 0x5599e4986f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.131+0000 7f584ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.131+0000 7f584ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48002/0 (socket says 172.21.15.57:48002) 2024-09-12T03:07:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.131+0000 7f584ea00640 1 -- 172.21.15.57:0/2794029691 learned_addr learned my addr 172.21.15.57:0/2794029691 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.132+0000 7f584ea00640 1 -- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 msgr2=0x5599e4a4b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:07:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.132+0000 7f584ea00640 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a4b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.132+0000 7f584ea00640 1 -- 172.21.15.57:0/2794029691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 msgr2=0x5599e4a4b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.133+0000 7f584ea00640 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.133+0000 7f584ea00640 1 -- 172.21.15.57:0/2794029691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e4788620 con 0x5599e4a4bec0 2024-09-12T03:07:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.133+0000 7f584ea00640 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f583c009850 tx=0x7f583c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=708a0995aad00165 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.133+0000 7f5839600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:07:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.134+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c019050 con 0x5599e4a4bec0 2024-09-12T03:07:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.134+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f583c0035f0 con 0x5599e4a4bec0 2024-09-12T03:07:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.134+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c004230 con 0x5599e4a4bec0 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f583c0055f0 con 0x5599e4a4bec0 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 10 client.25384 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599e4a6e2a0 con 0x5599e4a4bec0 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c004230 con 0x5599e4a4bec0 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f583c0114a0 con 0x5599e4a4bec0 2024-09-12T03:07:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 10 client.25384 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 20 client.25384 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.135+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4a6f6b0 0x5599e4a71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.136+0000 7f58595ca740 4 client.25384 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.136+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e49b6830 con 0x5599e4a6f6b0 2024-09-12T03:07:20.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.136+0000 7f584f400640 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4a6f6b0 0x5599e4a71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.137+0000 7f584f400640 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4a6f6b0 0x5599e4a71aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc8e5453567f677 server_cookie=8f1d5d64e54a0c8a in_seq=0 out_seq=0 2024-09-12T03:07:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.137+0000 7f5839600640 10 client.25384 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.137+0000 7f5839600640 1 -- 172.21.15.57:0/2794029691 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599e49b6830 con 0x5599e4a6f6b0 2024-09-12T03:07:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.137+0000 7f5839600640 10 client.25384 handle_command_reply: tid=0 2024-09-12T03:07:20.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.138+0000 7f58595ca740 1 client.25384 shutdown 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.138+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 msgr2=0x5599e4a6b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.138+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f583c009850 tx=0x7f583c009880 comp rx=0 tx=0).stop 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.138+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 shutdown_connections 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.138+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4a6f6b0 0x5599e4a71aa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.139+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a4bec0 0x5599e4a6b4c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.139+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a4b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.139+0000 7f58595ca740 1 --2- 172.21.15.57:0/2794029691 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a43b00 0x5599e4a4b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.139+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 >> 172.21.15.57:0/2794029691 conn(0x5599e4a3f750 msgr2=0x5599e4a454b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.139+0000 7f5839600640 0 client.25384 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.140+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 shutdown_connections 2024-09-12T03:07:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.144+0000 7f58595ca740 1 -- 172.21.15.57:0/2794029691 wait complete. 2024-09-12T03:07:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.144+0000 7f58595ca740 20 client.25384 trim_cache size 0 max 16384 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 Processor -- start 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 -- start start 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a6b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 0x5599e4a3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5830012900 con 0x5599e4a4bec0 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f58300101d0 con 0x5599e4a4e3a0 2024-09-12T03:07:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.176+0000 7f58595ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f583000e020 con 0x5599e4a43b00 2024-09-12T03:07:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.177+0000 7f584ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 0x5599e4a3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60574/0 (socket says 172.21.15.57:60574) 2024-09-12T03:07:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 -- 172.21.15.57:0/1194997980 learned_addr learned my addr 172.21.15.57:0/1194997980 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 0x5599e4a3e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57012/0 (socket says 172.21.15.57:57012) 2024-09-12T03:07:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a6b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 -- 172.21.15.57:0/1194997980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 msgr2=0x5599e4a6b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a6b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 -- 172.21.15.57:0/1194997980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 msgr2=0x5599e4a3e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 0x5599e4a3e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.178+0000 7f584ea00640 1 -- 172.21.15.57:0/1194997980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e4a42f60 con 0x5599e4a4e3a0 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.179+0000 7f584ea00640 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f583c005e00 tx=0x7f583c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e613ab9892bde1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.179+0000 7f583be00640 1 -- 172.21.15.57:0/1194997980 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c019040 con 0x5599e4a4e3a0 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.179+0000 7f583be00640 1 -- 172.21.15.57:0/1194997980 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c039040 con 0x5599e4a4e3a0 2024-09-12T03:07:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.179+0000 7f583be00640 1 -- 172.21.15.57:0/1194997980 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c003d50 con 0x5599e4a4e3a0 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.180+0000 7f58595ca740 1 -- 172.21.15.57:0/1194997980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 msgr2=0x5599e4a77070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.180+0000 7f58595ca740 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f583c005e00 tx=0x7f583c009ea0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 -- 172.21.15.57:0/1194997980 shutdown_connections 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4e3a0 0x5599e4a77070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4bec0 0x5599e4a3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 --2- 172.21.15.57:0/1194997980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 -- 172.21.15.57:0/1194997980 >> 172.21.15.57:0/1194997980 conn(0x5599e4a3f750 msgr2=0x5599e4a3e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 -- 172.21.15.57:0/1194997980 shutdown_connections 2024-09-12T03:07:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.181+0000 7f58595ca740 1 -- 172.21.15.57:0/1194997980 wait complete. 2024-09-12T03:07:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.182+0000 7f58595ca740 1 -- start start 2024-09-12T03:07:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.182+0000 7f58595ca740 1 Processor -- start 2024-09-12T03:07:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.182+0000 7f58595ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.182+0000 7f58595ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 0x5599e4a488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f58595ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4e3a0 0x5599e4a48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f58595ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f583000e690 con 0x5599e4a4e3a0 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f58595ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f583000fb00 con 0x5599e4a4bec0 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f58595ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f583000ed60 con 0x5599e4a43b00 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f58595ca740 10 client.0 _ll_register_callbacks cb 0x7f583c009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48020/0 (socket says 172.21.15.57:48020) 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 -- 172.21.15.57:0/3078413323 learned_addr learned my addr 172.21.15.57:0/3078413323 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584f400640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 0x5599e4a488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4e3a0 0x5599e4a48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 -- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 msgr2=0x5599e4a488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 0x5599e4a488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 -- 172.21.15.57:0/3078413323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4e3a0 msgr2=0x5599e4a48df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4e3a0 0x5599e4a48df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584ea00640 1 -- 172.21.15.57:0/3078413323 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e4ad9040 con 0x5599e4a43b00 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.183+0000 7f584f400640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 0x5599e4a488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f584ea00640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f583c013e30 tx=0x7f583c0131d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcdda0f94173a7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f5839600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c019040 con 0x5599e4a43b00 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c039040 con 0x5599e4a43b00 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f583c0035f0 con 0x5599e4a43b00 2024-09-12T03:07:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.184+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f583c003810 con 0x5599e4a43b00 2024-09-12T03:07:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.186+0000 7f58595ca740 10 client.25390 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.186+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599e4ada290 con 0x5599e4a43b00 2024-09-12T03:07:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.186+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c039040 con 0x5599e4a43b00 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f583c028c30 con 0x5599e4a43b00 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 10 client.25390 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 20 client.25390 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4ada8f0 0x5599e4adcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 4 client.25390 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.187+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599e49b0b80 con 0x5599e4ada8f0 2024-09-12T03:07:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.188+0000 7f584f400640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4ada8f0 0x5599e4adcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.188+0000 7f5839600640 10 client.25390 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.188+0000 7f584f400640 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4ada8f0 0x5599e4adcd10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8f01b08f35861e server_cookie=6c7d46ddf77bcdb3 in_seq=0 out_seq=0 2024-09-12T03:07:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.197+0000 7f5839600640 1 -- 172.21.15.57:0/3078413323 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599e49b0b80 con 0x5599e4ada8f0 2024-09-12T03:07:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.197+0000 7f5839600640 10 client.25390 handle_command_reply: tid=0 2024-09-12T03:07:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.197+0000 7f58595ca740 1 client.25390 shutdown 2024-09-12T03:07:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 msgr2=0x5599e4a48370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f583c013e30 tx=0x7f583c0131d0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 shutdown_connections 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5599e4ada8f0 0x5599e4adcd10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5599e4a4e3a0 0x5599e4a48df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5599e4a4bec0 0x5599e4a488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 --2- 172.21.15.57:0/3078413323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5599e4a43b00 0x5599e4a48370 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f5839600640 0 client.25390 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 >> 172.21.15.57:0/3078413323 conn(0x5599e4a3f750 msgr2=0x5599e4a4d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 shutdown_connections 2024-09-12T03:07:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.198+0000 7f58595ca740 1 -- 172.21.15.57:0/3078413323 wait complete. 2024-09-12T03:07:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.199+0000 7f58595ca740 20 client.25390 trim_cache size 0 max 16384 2024-09-12T03:07:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:07:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:07:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:07:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:07:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.199+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 msgr2=0x7f58340a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.199+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 0x7f58340a58e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f58480ccb50 tx=0x7f58480d5240 comp rx=0 tx=0).stop 2024-09-12T03:07:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.199+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 msgr2=0x7f585019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.199+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f584800eaa0 tx=0x7f584800ef60 comp rx=0 tx=0).stop 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 shutdown_connections 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f585019e9a0 0x7f585019ee50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f58340a3420 0x7f58340a58e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5850112ae0 0x7f58501a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 --2- 172.21.15.57:0/2175212969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5850074eb0 0x7f58501a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 >> 172.21.15.57:0/2175212969 conn(0x7f58500fd7d0 msgr2=0x7f58500feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 shutdown_connections 2024-09-12T03:07:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:20.200+0000 7f5856600640 1 -- 172.21.15.57:0/2175212969 wait complete. 2024-09-12T03:07:21.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:20 smithi204 ceph-mon[35013]: pgmap v45: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 233 MiB/s wr, 74 op/s 2024-09-12T03:07:21.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:20 smithi181 ceph-mon[35320]: pgmap v45: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 233 MiB/s wr, 74 op/s 2024-09-12T03:07:21.366 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:20 smithi057 ceph-mon[30361]: pgmap v45: 129 pgs: 129 active+clean; 3.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 233 MiB/s wr, 74 op/s 2024-09-12T03:07:23.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:22 smithi204 ceph-mon[35013]: pgmap v46: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 332 MiB/s wr, 104 op/s 2024-09-12T03:07:23.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:22 smithi181 ceph-mon[35320]: pgmap v46: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 332 MiB/s wr, 104 op/s 2024-09-12T03:07:23.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:22 smithi057 ceph-mon[30361]: pgmap v46: 129 pgs: 129 active+clean; 4.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 332 MiB/s wr, 104 op/s 2024-09-12T03:07:25.175 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:24 smithi057 ceph-mon[30361]: pgmap v47: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-12T03:07:25.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:24 smithi181 ceph-mon[35320]: pgmap v47: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-12T03:07:25.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:24 smithi204 ceph-mon[35013]: pgmap v47: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 71 op/s 2024-09-12T03:07:27.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:26 smithi057 ceph-mon[30361]: pgmap v48: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 MiB/s wr, 68 op/s 2024-09-12T03:07:27.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:26 smithi181 ceph-mon[35320]: pgmap v48: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 MiB/s wr, 68 op/s 2024-09-12T03:07:27.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:26 smithi204 ceph-mon[35013]: pgmap v48: 129 pgs: 129 active+clean; 4.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 MiB/s wr, 68 op/s 2024-09-12T03:07:29.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:28 smithi181 ceph-mon[35320]: pgmap v49: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 352 MiB/s wr, 102 op/s 2024-09-12T03:07:29.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:28 smithi181 ceph-mon[35320]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-09-12T03:07:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:28 smithi057 ceph-mon[30361]: pgmap v49: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 352 MiB/s wr, 102 op/s 2024-09-12T03:07:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:28 smithi057 ceph-mon[30361]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-09-12T03:07:29.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:28 smithi204 ceph-mon[35013]: pgmap v49: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 352 MiB/s wr, 102 op/s 2024-09-12T03:07:29.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:28 smithi204 ceph-mon[35013]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-09-12T03:07:31.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:30 smithi181 ceph-mon[35320]: pgmap v50: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 235 MiB/s wr, 69 op/s 2024-09-12T03:07:31.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:30 smithi057 ceph-mon[30361]: pgmap v50: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 235 MiB/s wr, 69 op/s 2024-09-12T03:07:31.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:30 smithi204 ceph-mon[35013]: pgmap v50: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 235 MiB/s wr, 69 op/s 2024-09-12T03:07:33.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:32 smithi181 ceph-mon[35320]: pgmap v51: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 333 MiB/s wr, 96 op/s 2024-09-12T03:07:33.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:32 smithi057 ceph-mon[30361]: pgmap v51: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 333 MiB/s wr, 96 op/s 2024-09-12T03:07:33.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:32 smithi204 ceph-mon[35013]: pgmap v51: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 333 MiB/s wr, 96 op/s 2024-09-12T03:07:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.399+0000 7f64e4c00640 1 -- 172.21.15.57:0/366715287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 msgr2=0x7f64e0115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.399+0000 7f64e4c00640 1 --2- 172.21.15.57:0/366715287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e0115ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009be0 tx=0x7f64d802a770 comp rx=0 tx=0).stop 2024-09-12T03:07:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.400+0000 7f64e4c00640 1 -- 172.21.15.57:0/366715287 shutdown_connections 2024-09-12T03:07:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.400+0000 7f64e4c00640 1 --2- 172.21.15.57:0/366715287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e0115ca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.400+0000 7f64e4c00640 1 --2- 172.21.15.57:0/366715287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e0113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.400+0000 7f64e4c00640 1 --2- 172.21.15.57:0/366715287 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.400+0000 7f64e4c00640 1 -- 172.21.15.57:0/366715287 >> 172.21.15.57:0/366715287 conn(0x7f64e006cbf0 msgr2=0x7f64e006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.401+0000 7f64e4c00640 1 -- 172.21.15.57:0/366715287 shutdown_connections 2024-09-12T03:07:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.401+0000 7f64e4c00640 1 -- 172.21.15.57:0/366715287 wait complete. 2024-09-12T03:07:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.403+0000 7f64e4c00640 1 Processor -- start 2024-09-12T03:07:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.403+0000 7f64e4c00640 1 -- start start 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e010b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e010be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01193a0 con 0x7f64e010a300 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0119220 con 0x7f64e010ac20 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64e4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0119520 con 0x7f64e01138b0 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e010b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e010b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:42810/0 (socket says 172.21.15.57:42810) 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64dea00640 1 -- 172.21.15.57:0/1738285951 learned_addr learned my addr 172.21.15.57:0/1738285951 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.404+0000 7f64df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64dfe00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e010be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64df400640 1 -- 172.21.15.57:0/1738285951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 msgr2=0x7f64e010be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64df400640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e010be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64df400640 1 -- 172.21.15.57:0/1738285951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 msgr2=0x7f64e010b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64df400640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e010b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.405+0000 7f64df400640 1 -- 172.21.15.57:0/1738285951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e01b3190 con 0x7f64e010a300 2024-09-12T03:07:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.406+0000 7f64df400640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010b3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c9a0 tx=0x7f64cc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.406+0000 7f64d7e00640 1 -- 172.21.15.57:0/1738285951 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc007bb0 con 0x7f64e010a300 2024-09-12T03:07:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.406+0000 7f64d7e00640 1 -- 172.21.15.57:0/1738285951 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64cc007d50 con 0x7f64e010a300 2024-09-12T03:07:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.406+0000 7f64d7e00640 1 -- 172.21.15.57:0/1738285951 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc018960 con 0x7f64e010a300 2024-09-12T03:07:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.407+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e01b3480 con 0x7f64e010a300 2024-09-12T03:07:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.407+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e01b3980 con 0x7f64e010a300 2024-09-12T03:07:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.416+0000 7f64d7e00640 1 -- 172.21.15.57:0/1738285951 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f64cc018b00 con 0x7f64e010a300 2024-09-12T03:07:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.418+0000 7f64d7e00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 0x7f64c40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.418+0000 7f64d7e00640 1 -- 172.21.15.57:0/1738285951 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f64cc0cfbf0 con 0x7f64e010a300 2024-09-12T03:07:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.419+0000 7f64e9070740 1 Processor -- start 2024-09-12T03:07:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.420+0000 7f64dea00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 0x7f64c40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.420+0000 7f64e9070740 1 -- start start 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea19570 0x561d5ea1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d5e974100 con 0x561d5ea0e3b0 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d5e91c1b0 con 0x561d5ea0ecd0 2024-09-12T03:07:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.422+0000 7f64e9070740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561d5e9700c0 con 0x561d5ea19570 2024-09-12T03:07:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:42824/0 (socket says 172.21.15.57:42824) 2024-09-12T03:07:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 -- 172.21.15.57:0/2957556074 learned_addr learned my addr 172.21.15.57:0/2957556074 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dfe00640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea19570 0x561d5ea1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 -- 172.21.15.57:0/2957556074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea19570 msgr2=0x561d5ea1b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea19570 0x561d5ea1b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64dea00640 1 -- 172.21.15.57:0/2957556074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 msgr2=0x561d5ea0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.423+0000 7f64df400640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.424+0000 7f64dea00640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.424+0000 7f64dea00640 1 -- 172.21.15.57:0/2957556074 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5e687210 con 0x561d5ea0ecd0 2024-09-12T03:07:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.425+0000 7f64dea00640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64d0014850 tx=0x7f64d001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8807c1a44a573c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.425+0000 7f64df400640 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.427+0000 7f64cbe00640 1 -- 172.21.15.57:0/2957556074 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0005020 con 0x561d5ea0ecd0 2024-09-12T03:07:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.427+0000 7f64cbe00640 1 -- 172.21.15.57:0/2957556074 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d00107d0 con 0x561d5ea0ecd0 2024-09-12T03:07:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.427+0000 7f64cbe00640 1 -- 172.21.15.57:0/2957556074 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00049e0 con 0x561d5ea0ecd0 2024-09-12T03:07:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.427+0000 7f64dea00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 0x7f64c40a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64d000bfd0 tx=0x7f64d000bb50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 -- 172.21.15.57:0/2957556074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 msgr2=0x561d5ea19030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64d0014850 tx=0x7f64d001c000 comp rx=0 tx=0).stop 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 -- 172.21.15.57:0/2957556074 shutdown_connections 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea19570 0x561d5ea1b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea19030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 --2- 172.21.15.57:0/2957556074 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.429+0000 7f64e9070740 1 -- 172.21.15.57:0/2957556074 >> 172.21.15.57:0/2957556074 conn(0x561d5ea0a920 msgr2=0x561d5ea0ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.430+0000 7f64e9070740 1 -- 172.21.15.57:0/2957556074 shutdown_connections 2024-09-12T03:07:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.430+0000 7f64e9070740 1 -- 172.21.15.57:0/2957556074 wait complete. 2024-09-12T03:07:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.432+0000 7f64e9070740 1 -- start start 2024-09-12T03:07:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.432+0000 7f64e9070740 1 Processor -- start 2024-09-12T03:07:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.435+0000 7f64e9070740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 0x561d5ea166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.435+0000 7f64e9070740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.435+0000 7f64e9070740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea17130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.436+0000 7f64e9070740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561d5ea1bf90 con 0x561d5ea19570 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64e9070740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561d5e8fd090 con 0x561d5ea0e3b0 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64e9070740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561d5ea1c110 con 0x561d5ea0ecd0 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64df400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 0x561d5ea166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64e9070740 10 client.0 _ll_register_callbacks cb 0x561d5e951f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64df400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 0x561d5ea166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:42832/0 (socket says 172.21.15.57:42832) 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64df400640 1 -- 172.21.15.57:0/96749634 learned_addr learned my addr 172.21.15.57:0/96749634 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64dfe00640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea17130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.437+0000 7f64dea00640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.438+0000 7f64dfe00640 1 -- 172.21.15.57:0/96749634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 msgr2=0x561d5ea16bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.438+0000 7f64dfe00640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.438+0000 7f64dfe00640 1 -- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 msgr2=0x561d5ea166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.438+0000 7f64dfe00640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 0x561d5ea166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.438+0000 7f64dfe00640 1 -- 172.21.15.57:0/96749634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5e753620 con 0x561d5ea19570 2024-09-12T03:07:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.439+0000 7f64dfe00640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea17130 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64d8034040 tx=0x7f64d8030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c672e70e29765018 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.439+0000 7f64c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:07:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.439+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8041070 con 0x561d5ea19570 2024-09-12T03:07:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.439+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d00107d0 con 0x561d5ea19570 2024-09-12T03:07:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.441+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f64d8003970 con 0x561d5ea19570 2024-09-12T03:07:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.441+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64d804b050 con 0x561d5ea19570 2024-09-12T03:07:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.441+0000 7f64e9070740 10 client.15330 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.441+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d5ea27da0 con 0x561d5ea19570 2024-09-12T03:07:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d8030ea0 con 0x561d5ea19570 2024-09-12T03:07:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64d804e020 con 0x561d5ea19570 2024-09-12T03:07:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64e9070740 10 client.15330 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64e9070740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64e9070740 20 client.15330 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.442+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea2ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.443+0000 7f64e9070740 4 client.15330 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.443+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64d00107d0 con 0x561d5ea29670 2024-09-12T03:07:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.443+0000 7f64df400640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea2ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.443+0000 7f64c8c00640 10 client.15330 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.443+0000 7f64df400640 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea2ba60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa036ba08514107b server_cookie=8e450d0c75a58897 in_seq=0 out_seq=0 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.444+0000 7f64c8c00640 1 -- 172.21.15.57:0/96749634 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f64d00107d0 con 0x561d5ea29670 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.444+0000 7f64c8c00640 10 client.15330 handle_command_reply: tid=0 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.444+0000 7f64e9070740 1 client.15330 shutdown 2024-09-12T03:07:33.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.444+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 msgr2=0x561d5ea17130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.444+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea17130 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f64d8034040 tx=0x7f64d8030430 comp rx=0 tx=0).stop 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 shutdown_connections 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea2ba60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64c8c00640 0 client.15330 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea17130 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 --2- 172.21.15.57:0/96749634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0e3b0 0x561d5ea166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 >> 172.21.15.57:0/96749634 conn(0x561d5ea0a920 msgr2=0x561d5ea10680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 shutdown_connections 2024-09-12T03:07:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 1 -- 172.21.15.57:0/96749634 wait complete. 2024-09-12T03:07:33.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.445+0000 7f64e9070740 20 client.15330 trim_cache size 0 max 16384 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 Processor -- start 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 -- start start 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0e3b0 0x561d5ea10e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea13970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01c05a0 con 0x561d5ea19570 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01bde70 con 0x561d5ea0ecd0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.463+0000 7f64e9070740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01bbc60 con 0x561d5ea0e3b0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:42840/0 (socket says 172.21.15.57:42840) 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 -- 172.21.15.57:0/1321166776 learned_addr learned my addr 172.21.15.57:0/1321166776 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64df400640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0e3b0 0x561d5ea10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dfe00640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea13970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 -- 172.21.15.57:0/1321166776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0e3b0 msgr2=0x561d5ea10e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0e3b0 0x561d5ea10e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 -- 172.21.15.57:0/1321166776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 msgr2=0x561d5ea13970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea13970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 -- 172.21.15.57:0/1321166776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5ea19160 con 0x561d5ea0ecd0 2024-09-12T03:07:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dfe00640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea13970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:07:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.464+0000 7f64dea00640 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64d000b770 tx=0x7f64d000b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bd5a41a5ed47ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.465+0000 7f64cb400640 1 -- 172.21.15.57:0/1321166776 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d002e020 con 0x561d5ea0ecd0 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.465+0000 7f64cb400640 1 -- 172.21.15.57:0/1321166776 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d0025040 con 0x561d5ea0ecd0 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.465+0000 7f64cb400640 1 -- 172.21.15.57:0/1321166776 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0011bd0 con 0x561d5ea0ecd0 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 -- 172.21.15.57:0/1321166776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 msgr2=0x561d5ea16d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64d000b770 tx=0x7f64d000b890 comp rx=0 tx=0).stop 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 -- 172.21.15.57:0/1321166776 shutdown_connections 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea19570 0x561d5ea13970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea0ecd0 0x561d5ea16d80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 --2- 172.21.15.57:0/1321166776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0e3b0 0x561d5ea10e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 -- 172.21.15.57:0/1321166776 >> 172.21.15.57:0/1321166776 conn(0x561d5ea0a920 msgr2=0x561d5ea16670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 -- 172.21.15.57:0/1321166776 shutdown_connections 2024-09-12T03:07:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.466+0000 7f64e9070740 1 -- 172.21.15.57:0/1321166776 wait complete. 2024-09-12T03:07:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.467+0000 7f64e9070740 1 -- start start 2024-09-12T03:07:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.467+0000 7f64e9070740 1 Processor -- start 2024-09-12T03:07:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea19570 0x561d5ea2f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01bc330 con 0x561d5ea0e3b0 2024-09-12T03:07:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01bd7a0 con 0x561d5ea19570 2024-09-12T03:07:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64e01bca00 con 0x561d5ea0ecd0 2024-09-12T03:07:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64e9070740 10 client.0 _ll_register_callbacks cb 0x7f64d8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64dea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64dea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:53284/0 (socket says 172.21.15.57:53284) 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64dea00640 1 -- 172.21.15.57:0/1974012850 learned_addr learned my addr 172.21.15.57:0/1974012850 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.468+0000 7f64dfe00640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea19570 0x561d5ea2f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 -- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea19570 msgr2=0x561d5ea2f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea19570 0x561d5ea2f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 -- 172.21.15.57:0/1974012850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 msgr2=0x561d5ea318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea318c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 -- 172.21.15.57:0/1974012850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5ea2f920 con 0x561d5ea0ecd0 2024-09-12T03:07:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64dea00640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64d00107a0 tx=0x7f64d00141f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dcb14ccc859acf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d002e050 con 0x561d5ea0ecd0 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f64d0010d90 con 0x561d5ea0ecd0 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d0025040 con 0x561d5ea0ecd0 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64d0011850 con 0x561d5ea0ecd0 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64e9070740 10 client.25405 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.469+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d5ea94180 con 0x561d5ea0ecd0 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d0025040 con 0x561d5ea0ecd0 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64d003cb30 con 0x561d5ea0ecd0 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 10 client.25405 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 20 client.25405 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea96570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 4 client.25405 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:07:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d5e981830 con 0x561d5ea29670 2024-09-12T03:07:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.470+0000 7f64df400640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea96570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.471+0000 7f64df400640 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea96570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0ff18de53b4153 server_cookie=7f3ec32fe607c6a4 in_seq=0 out_seq=0 2024-09-12T03:07:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.471+0000 7f64c8c00640 10 client.25405 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64c8c00640 1 -- 172.21.15.57:0/1974012850 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d5e981830 con 0x561d5ea29670 2024-09-12T03:07:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64c8c00640 10 client.25405 handle_command_reply: tid=0 2024-09-12T03:07:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 client.25405 shutdown 2024-09-12T03:07:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 msgr2=0x561d5ea2ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64d00107a0 tx=0x7f64d00141f0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 shutdown_connections 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561d5ea29670 0x561d5ea96570 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561d5ea19570 0x561d5ea2f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561d5ea0ecd0 0x561d5ea2ec00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 --2- 172.21.15.57:0/1974012850 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561d5ea0e3b0 0x561d5ea318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 >> 172.21.15.57:0/1974012850 conn(0x561d5ea0a920 msgr2=0x561d5ea24bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64c8c00640 0 client.25405 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 shutdown_connections 2024-09-12T03:07:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 1 -- 172.21.15.57:0/1974012850 wait complete. 2024-09-12T03:07:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.480+0000 7f64e9070740 20 client.25405 trim_cache size 0 max 16384 2024-09-12T03:07:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:07:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:07:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:07:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:07:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.481+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 msgr2=0x7f64c40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.481+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 0x7f64c40a5380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f64d000bfd0 tx=0x7f64d000bb50 comp rx=0 tx=0).stop 2024-09-12T03:07:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.481+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 msgr2=0x7f64e010b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.481+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010b3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c9a0 tx=0x7f64cc00ce60 comp rx=0 tx=0).stop 2024-09-12T03:07:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 shutdown_connections 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e01138b0 0x7f64e010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64e010ac20 0x7f64e010b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f64c40a2ec0 0x7f64c40a5380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 --2- 172.21.15.57:0/1738285951 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f64e010a300 0x7f64e010b3e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 >> 172.21.15.57:0/1738285951 conn(0x7f64e006cbf0 msgr2=0x7f64e01091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.482+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 shutdown_connections 2024-09-12T03:07:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:33.486+0000 7f64e4c00640 1 -- 172.21.15.57:0/1738285951 wait complete. 2024-09-12T03:07:35.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:34 smithi181 ceph-mon[35320]: pgmap v52: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 247 MiB/s wr, 70 op/s 2024-09-12T03:07:35.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:34 smithi057 ceph-mon[30361]: pgmap v52: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 247 MiB/s wr, 70 op/s 2024-09-12T03:07:35.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:34 smithi204 ceph-mon[35013]: pgmap v52: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 247 MiB/s wr, 70 op/s 2024-09-12T03:07:37.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:36 smithi057 ceph-mon[30361]: pgmap v53: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 230 MiB/s wr, 65 op/s 2024-09-12T03:07:37.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:36 smithi204 ceph-mon[35013]: pgmap v53: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 230 MiB/s wr, 65 op/s 2024-09-12T03:07:37.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:36 smithi181 ceph-mon[35320]: pgmap v53: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 230 MiB/s wr, 65 op/s 2024-09-12T03:07:39.377 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:38 smithi057 ceph-mon[30361]: pgmap v54: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 345 MiB/s wr, 98 op/s 2024-09-12T03:07:39.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:38 smithi204 ceph-mon[35013]: pgmap v54: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 345 MiB/s wr, 98 op/s 2024-09-12T03:07:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:38 smithi181 ceph-mon[35320]: pgmap v54: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 345 MiB/s wr, 98 op/s 2024-09-12T03:07:41.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:40 smithi057 ceph-mon[30361]: pgmap v55: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 226 MiB/s wr, 64 op/s 2024-09-12T03:07:41.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:40 smithi204 ceph-mon[35013]: pgmap v55: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 226 MiB/s wr, 64 op/s 2024-09-12T03:07:41.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:40 smithi181 ceph-mon[35320]: pgmap v55: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 226 MiB/s wr, 64 op/s 2024-09-12T03:07:43.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:42 smithi057 ceph-mon[30361]: pgmap v56: 129 pgs: 129 active+clean; 9.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 332 MiB/s wr, 94 op/s 2024-09-12T03:07:43.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:42 smithi204 ceph-mon[35013]: pgmap v56: 129 pgs: 129 active+clean; 9.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 332 MiB/s wr, 94 op/s 2024-09-12T03:07:43.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:43 smithi181 ceph-mon[35320]: pgmap v56: 129 pgs: 129 active+clean; 9.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 332 MiB/s wr, 94 op/s 2024-09-12T03:07:45.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:45 smithi057 ceph-mon[30361]: pgmap v57: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 248 MiB/s wr, 70 op/s 2024-09-12T03:07:45.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:45 smithi204 ceph-mon[35013]: pgmap v57: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 248 MiB/s wr, 70 op/s 2024-09-12T03:07:45.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:45 smithi181 ceph-mon[35320]: pgmap v57: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 248 MiB/s wr, 70 op/s 2024-09-12T03:07:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.744+0000 7fad27c00640 1 -- 172.21.15.57:0/2492339232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad20109b10 msgr2=0x7fad20116960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.744+0000 7fad27c00640 1 --2- 172.21.15.57:0/2492339232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad20109b10 0x7fad20116960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad10009900 tx=0x7fad1002a410 comp rx=0 tx=0).stop 2024-09-12T03:07:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 -- 172.21.15.57:0/2492339232 shutdown_connections 2024-09-12T03:07:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 --2- 172.21.15.57:0/2492339232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad20119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 --2- 172.21.15.57:0/2492339232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad20109b10 0x7fad20116960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 --2- 172.21.15.57:0/2492339232 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad201091f0 0x7fad201095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 -- 172.21.15.57:0/2492339232 >> 172.21.15.57:0/2492339232 conn(0x7fad2006ce00 msgr2=0x7fad2006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.748+0000 7fad27c00640 1 -- 172.21.15.57:0/2492339232 shutdown_connections 2024-09-12T03:07:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.749+0000 7fad27c00640 1 -- 172.21.15.57:0/2492339232 wait complete. 2024-09-12T03:07:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.750+0000 7fad27c00640 1 Processor -- start 2024-09-12T03:07:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.750+0000 7fad27c00640 1 -- start start 2024-09-12T03:07:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.751+0000 7fad27c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 0x7fad201a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.752+0000 7fad27c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 0x7fad201a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad27c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.752+0000 7fad25200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 0x7fad201a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad25200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 0x7fad201a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60048/0 (socket says 172.21.15.57:60048) 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad27c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fad2011c9b0 con 0x7fad20109b10 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad27c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad2011c830 con 0x7fad20116ea0 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad27c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad2011cb30 con 0x7fad201091f0 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad25c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad25c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38422/0 (socket says 172.21.15.57:38422) 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.753+0000 7fad25c00640 1 -- 172.21.15.57:0/2320574780 learned_addr learned my addr 172.21.15.57:0/2320574780 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad1fe00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 0x7fad201a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad25c00640 1 -- 172.21.15.57:0/2320574780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 msgr2=0x7fad201a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad25c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 0x7fad201a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad25c00640 1 -- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 msgr2=0x7fad201a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad25c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 0x7fad201a32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.754+0000 7fad25c00640 1 -- 172.21.15.57:0/2320574780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad201a7d70 con 0x7fad20116ea0 2024-09-12T03:07:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.755+0000 7fad1fe00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 0x7fad201a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:07:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.756+0000 7fad25c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c970 tx=0x7fad0c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.756+0000 7fad1d600640 1 -- 172.21.15.57:0/2320574780 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c007bb0 con 0x7fad20116ea0 2024-09-12T03:07:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.757+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad201a8060 con 0x7fad20116ea0 2024-09-12T03:07:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.758+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201af8f0 con 0x7fad20116ea0 2024-09-12T03:07:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.758+0000 7fad1d600640 1 -- 172.21.15.57:0/2320574780 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0c007d50 con 0x7fad20116ea0 2024-09-12T03:07:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.758+0000 7fad1d600640 1 -- 172.21.15.57:0/2320574780 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c018880 con 0x7fad20116ea0 2024-09-12T03:07:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.771+0000 7fad1d600640 1 -- 172.21.15.57:0/2320574780 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fad0c00f450 con 0x7fad20116ea0 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.773+0000 7fad2ac68740 1 Processor -- start 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.773+0000 7fad2ac68740 1 -- start start 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x55794296efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5579428bf100 con 0x55794296cbb0 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5579428671b0 con 0x5579429590e0 2024-09-12T03:07:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.774+0000 7fad2ac68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5579428bb0c0 con 0x557942959a00 2024-09-12T03:07:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.776+0000 7fad25200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.776+0000 7fad25200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60058/0 (socket says 172.21.15.57:60058) 2024-09-12T03:07:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.776+0000 7fad25200640 1 -- 172.21.15.57:0/2183653098 learned_addr learned my addr 172.21.15.57:0/2183653098 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.776+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x55794296efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.777+0000 7fad1fe00640 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.777+0000 7fad1d600640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 0x7fad040a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.778+0000 7fad25200640 1 -- 172.21.15.57:0/2183653098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 msgr2=0x5579429594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.778+0000 7fad25200640 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.778+0000 7fad25200640 1 -- 172.21.15.57:0/2183653098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 msgr2=0x55794296efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.778+0000 7fad25200640 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x55794296efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.778+0000 7fad25200640 1 -- 172.21.15.57:0/2183653098 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579425d2210 con 0x557942959a00 2024-09-12T03:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.779+0000 7fad1d600640 1 -- 172.21.15.57:0/2320574780 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fad0c0cf760 con 0x7fad20116ea0 2024-09-12T03:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.779+0000 7fad25200640 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad18009fa0 tx=0x7fad1800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e0d3a5d54bd986 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.780+0000 7fad16000640 1 -- 172.21.15.57:0/2183653098 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1801a070 con 0x557942959a00 2024-09-12T03:07:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.780+0000 7fad16000640 1 -- 172.21.15.57:0/2183653098 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad180158f0 con 0x557942959a00 2024-09-12T03:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.782+0000 7fad2ac68740 1 -- 172.21.15.57:0/2183653098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 msgr2=0x55794296c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.782+0000 7fad25c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 0x7fad040a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.782+0000 7fad16000640 1 -- 172.21.15.57:0/2183653098 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1801a070 con 0x557942959a00 2024-09-12T03:07:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.782+0000 7fad2ac68740 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad18009fa0 tx=0x7fad1800c8b0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.786+0000 7fad2ac68740 1 -- 172.21.15.57:0/2183653098 shutdown_connections 2024-09-12T03:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.787+0000 7fad2ac68740 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x55794296efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.787+0000 7fad2ac68740 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x55794296c670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.787+0000 7fad2ac68740 1 --2- 172.21.15.57:0/2183653098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.787+0000 7fad2ac68740 1 -- 172.21.15.57:0/2183653098 >> 172.21.15.57:0/2183653098 conn(0x557942955750 msgr2=0x557942955b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.790+0000 7fad25c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 0x7fad040a5330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad0c007590 tx=0x7fad0c0cfa60 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.790+0000 7fad2ac68740 1 -- 172.21.15.57:0/2183653098 shutdown_connections 2024-09-12T03:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.790+0000 7fad2ac68740 1 -- 172.21.15.57:0/2183653098 wait complete. 2024-09-12T03:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.791+0000 7fad2ac68740 1 -- start start 2024-09-12T03:07:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.791+0000 7fad2ac68740 1 Processor -- start 2024-09-12T03:07:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.793+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.793+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x557942966c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad25200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x557942966c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad25200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x557942966c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42104/0 (socket says 172.21.15.57:42104) 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38452/0 (socket says 172.21.15.57:38452) 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad25200640 1 -- 172.21.15.57:0/210449009 learned_addr learned my addr 172.21.15.57:0/210449009 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.794+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794296c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.795+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55794295c180 con 0x557942959a00 2024-09-12T03:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.795+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x557942848090 con 0x5579429590e0 2024-09-12T03:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.795+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55794295c300 con 0x55794296cbb0 2024-09-12T03:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794296c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad2ac68740 10 client.0 _ll_register_callbacks cb 0x55794289cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 -- 172.21.15.57:0/210449009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 msgr2=0x55794296c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794296c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 -- 172.21.15.57:0/210449009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 msgr2=0x557942966c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x557942966c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.796+0000 7fad1fe00640 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794269e620 con 0x5579429590e0 2024-09-12T03:07:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.797+0000 7fad1fe00640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad10009850 tx=0x7fad10005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4318ed9d26f4cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.797+0000 7fad0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:07:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.799+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10035070 con 0x5579429590e0 2024-09-12T03:07:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.799+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fad10004980 con 0x5579429590e0 2024-09-12T03:07:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.800+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad180158f0 con 0x5579429590e0 2024-09-12T03:07:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.800+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad10004b80 con 0x5579429590e0 2024-09-12T03:07:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.800+0000 7fad2ac68740 10 client.25400 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.801+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557942983d20 con 0x5579429590e0 2024-09-12T03:07:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.801+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad180158f0 con 0x5579429590e0 2024-09-12T03:07:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fad10003710 con 0x5579429590e0 2024-09-12T03:07:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad2ac68740 10 client.25400 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad2ac68740 10 client.25400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad2ac68740 20 client.25400 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429851e0 0x557942987670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.802+0000 7fad2ac68740 4 client.25400 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:07:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.803+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579428cc830 con 0x5579429851e0 2024-09-12T03:07:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.805+0000 7fad25200640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429851e0 0x557942987670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.807+0000 7fad25200640 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429851e0 0x557942987670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51d1c1d3fcfffff server_cookie=e0988dbd7ca46825 in_seq=0 out_seq=0 2024-09-12T03:07:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.807+0000 7fad0aa00640 10 client.25400 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.810+0000 7fad0aa00640 1 -- 172.21.15.57:0/210449009 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579428cc830 con 0x5579429851e0 2024-09-12T03:07:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.810+0000 7fad0aa00640 10 client.25400 handle_command_reply: tid=0 2024-09-12T03:07:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.810+0000 7fad2ac68740 1 client.25400 shutdown 2024-09-12T03:07:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 msgr2=0x5579429666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad10009850 tx=0x7fad10005d50 comp rx=0 tx=0).stop 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 shutdown_connections 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429851e0 0x557942987670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794296c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x557942966c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 --2- 172.21.15.57:0/210449009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579429590e0 0x5579429666d0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad10009850 tx=0x7fad10005d50 comp rx=0 tx=0).stop 2024-09-12T03:07:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.811+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 >> 172.21.15.57:0/210449009 conn(0x557942955750 msgr2=0x55794295b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.812+0000 7fad0aa00640 0 client.25400 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.812+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 shutdown_connections 2024-09-12T03:07:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.813+0000 7fad2ac68740 1 -- 172.21.15.57:0/210449009 wait complete. 2024-09-12T03:07:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.813+0000 7fad2ac68740 20 client.25400 trim_cache size 0 max 16384 2024-09-12T03:07:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.861+0000 7fad2ac68740 1 Processor -- start 2024-09-12T03:07:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.862+0000 7fad2ac68740 1 -- start start 2024-09-12T03:07:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x55794298cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794298d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fad00012870 con 0x557942959a00 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad00010140 con 0x557942960790 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad2ac68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad0000df90 con 0x55794296cbb0 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.863+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.864+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38460/0 (socket says 172.21.15.57:38460) 2024-09-12T03:07:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.865+0000 7fad1fe00640 1 -- 172.21.15.57:0/3211486911 learned_addr learned my addr 172.21.15.57:0/3211486911 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.865+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x55794298cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.866+0000 7fad1fe00640 1 -- 172.21.15.57:0/3211486911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 msgr2=0x55794298d480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.866+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794298d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.866+0000 7fad1fe00640 1 -- 172.21.15.57:0/3211486911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 msgr2=0x55794298cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.866+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x55794298cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.866+0000 7fad1fe00640 1 -- 172.21.15.57:0/3211486911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557942959830 con 0x557942960790 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.867+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad10031550 tx=0x7fad10000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fccd2b6210983f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.867+0000 7fad15600640 1 -- 172.21.15.57:0/3211486911 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10035040 con 0x557942960790 2024-09-12T03:07:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.868+0000 7fad15600640 1 -- 172.21.15.57:0/3211486911 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10003c60 con 0x557942960790 2024-09-12T03:07:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.869+0000 7fad2ac68740 1 -- 172.21.15.57:0/3211486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 msgr2=0x55794295ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.869+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad10031550 tx=0x7fad10000bc0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.874+0000 7fad2ac68740 1 -- 172.21.15.57:0/3211486911 shutdown_connections 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.874+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x55794295ece0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.874+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55794296cbb0 0x55794298d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.874+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3211486911 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x557942959a00 0x55794298cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.874+0000 7fad2ac68740 1 -- 172.21.15.57:0/3211486911 >> 172.21.15.57:0/3211486911 conn(0x557942955750 msgr2=0x557942954830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.875+0000 7fad2ac68740 1 -- 172.21.15.57:0/3211486911 shutdown_connections 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.875+0000 7fad2ac68740 1 -- 172.21.15.57:0/3211486911 wait complete. 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.876+0000 7fad2ac68740 1 -- start start 2024-09-12T03:07:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.876+0000 7fad2ac68740 1 Processor -- start 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x5579429994a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x557942996e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fad0000e600 con 0x55794296cbb0 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fad0000fa70 con 0x557942960790 2024-09-12T03:07:46.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad0000ecd0 con 0x557942959a00 2024-09-12T03:07:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.877+0000 7fad2ac68740 10 client.0 _ll_register_callbacks cb 0x7fad10009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:07:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.878+0000 7fad25200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.878+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x5579429994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.878+0000 7fad25200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38464/0 (socket says 172.21.15.57:38464) 2024-09-12T03:07:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.878+0000 7fad25200640 1 -- 172.21.15.57:0/3339839116 learned_addr learned my addr 172.21.15.57:0/3339839116 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:07:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.878+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x557942996e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.880+0000 7fad25200640 1 -- 172.21.15.57:0/3339839116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 msgr2=0x5579429994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.880+0000 7fad25200640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x5579429994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.880+0000 7fad25200640 1 -- 172.21.15.57:0/3339839116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 msgr2=0x557942996e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:07:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad25200640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x557942996e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad25200640 1 -- 172.21.15.57:0/3339839116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557942997650 con 0x557942960790 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x557942996e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad25200640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f40 tx=0x7fad18015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c323923b1c1fceb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1802f060 con 0x557942960790 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.881+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fad18015620 con 0x557942960790 2024-09-12T03:07:46.882 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.882+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad10003c60 con 0x557942960790 2024-09-12T03:07:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.882+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad1801a040 con 0x557942960790 2024-09-12T03:07:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.882+0000 7fad2ac68740 10 client.25406 fetch_fsmap learned FSMap version 28 2024-09-12T03:07:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.882+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579429eff20 con 0x557942960790 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad10003c60 con 0x557942960790 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fad1800a990 con 0x557942960790 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad2ac68740 10 client.25406 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad2ac68740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad2ac68740 20 client.25406 populate_metadata read hostname 'smithi057' 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.883+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429f03f0 0x5579429f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.884+0000 7fad2ac68740 4 client.25406 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:07:46.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.884+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579428c6b80 con 0x5579429f03f0 2024-09-12T03:07:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.884+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429f03f0 0x5579429f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:07:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.885+0000 7fad1fe00640 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429f03f0 0x5579429f2810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea49315b2d6d97c2 server_cookie=104d0b95a3332c16 in_seq=0 out_seq=0 2024-09-12T03:07:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.885+0000 7fad0aa00640 10 client.25406 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.904+0000 7fad0aa00640 1 -- 172.21.15.57:0/3339839116 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579428c6b80 con 0x5579429f03f0 2024-09-12T03:07:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.904+0000 7fad0aa00640 10 client.25406 handle_command_reply: tid=0 2024-09-12T03:07:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.905+0000 7fad2ac68740 1 client.25406 shutdown 2024-09-12T03:07:46.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.905+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 msgr2=0x557942996830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.905+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad18009f40 tx=0x7fad18015f90 comp rx=0 tx=0).stop 2024-09-12T03:07:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 shutdown_connections 2024-09-12T03:07:46.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5579429f03f0 0x5579429f2810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55794296cbb0 0x557942996e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557942960790 0x557942996830 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 --2- 172.21.15.57:0/3339839116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557942959a00 0x5579429994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 >> 172.21.15.57:0/3339839116 conn(0x557942955750 msgr2=0x55794295ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad0aa00640 0 client.25406 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 shutdown_connections 2024-09-12T03:07:46.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.906+0000 7fad2ac68740 1 -- 172.21.15.57:0/3339839116 wait complete. 2024-09-12T03:07:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.907+0000 7fad2ac68740 20 client.25406 trim_cache size 0 max 16384 2024-09-12T03:07:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:07:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:07:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:07:46.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.908+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 msgr2=0x7fad040a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.908+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 0x7fad040a5330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad0c007590 tx=0x7fad0c0cfa60 comp rx=0 tx=0).stop 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.908+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 msgr2=0x7fad201a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.908+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c970 tx=0x7fad0c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 shutdown_connections 2024-09-12T03:07:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad20116ea0 0x7fad201a7640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fad040a2e70 0x7fad040a5330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fad20109b10 0x7fad201a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 --2- 172.21.15.57:0/2320574780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad201091f0 0x7fad201a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 >> 172.21.15.57:0/2320574780 conn(0x7fad2006ce00 msgr2=0x7fad201172e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 shutdown_connections 2024-09-12T03:07:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:07:46.909+0000 7fad27c00640 1 -- 172.21.15.57:0/2320574780 wait complete. 2024-09-12T03:07:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:47 smithi057 ceph-mon[30361]: pgmap v58: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 235 MiB/s wr, 67 op/s 2024-09-12T03:07:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:47 smithi204 ceph-mon[35013]: pgmap v58: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 235 MiB/s wr, 67 op/s 2024-09-12T03:07:47.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:47 smithi181 ceph-mon[35320]: pgmap v58: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 235 MiB/s wr, 67 op/s 2024-09-12T03:07:49.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:49 smithi057 ceph-mon[30361]: pgmap v59: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 518 op/s 2024-09-12T03:07:49.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:49 smithi204 ceph-mon[35013]: pgmap v59: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 518 op/s 2024-09-12T03:07:49.452 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:49 smithi181 ceph-mon[35320]: pgmap v59: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 518 op/s 2024-09-12T03:07:51.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:51 smithi057 ceph-mon[30361]: pgmap v60: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 222 MiB/s wr, 484 op/s 2024-09-12T03:07:51.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:51 smithi204 ceph-mon[35013]: pgmap v60: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 222 MiB/s wr, 484 op/s 2024-09-12T03:07:51.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:51 smithi181 ceph-mon[35320]: pgmap v60: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 222 MiB/s wr, 484 op/s 2024-09-12T03:07:53.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:53 smithi057 ceph-mon[30361]: pgmap v61: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 300 MiB/s wr, 828 op/s 2024-09-12T03:07:53.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:53 smithi204 ceph-mon[35013]: pgmap v61: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 300 MiB/s wr, 828 op/s 2024-09-12T03:07:53.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:53 smithi181 ceph-mon[35320]: pgmap v61: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 300 MiB/s wr, 828 op/s 2024-09-12T03:07:55.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:55 smithi057 ceph-mon[30361]: pgmap v62: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 MiB/s wr, 919 op/s 2024-09-12T03:07:55.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:55 smithi204 ceph-mon[35013]: pgmap v62: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 MiB/s wr, 919 op/s 2024-09-12T03:07:55.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:55 smithi181 ceph-mon[35320]: pgmap v62: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 223 MiB/s wr, 919 op/s 2024-09-12T03:07:57.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:57 smithi057 ceph-mon[30361]: pgmap v63: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 209 MiB/s wr, 916 op/s 2024-09-12T03:07:57.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:57 smithi204 ceph-mon[35013]: pgmap v63: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 209 MiB/s wr, 916 op/s 2024-09-12T03:07:57.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:57 smithi181 ceph-mon[35320]: pgmap v63: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 209 MiB/s wr, 916 op/s 2024-09-12T03:07:59.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:07:59 smithi057 ceph-mon[30361]: pgmap v64: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 1.29k op/s 2024-09-12T03:07:59.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:07:59 smithi204 ceph-mon[35013]: pgmap v64: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 1.29k op/s 2024-09-12T03:07:59.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:07:59 smithi181 ceph-mon[35320]: pgmap v64: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 1.29k op/s 2024-09-12T03:08:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.142+0000 7f7ba3800640 1 -- 172.21.15.57:0/86668660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c10d940 msgr2=0x7f7b9c115460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.142+0000 7f7ba3800640 1 --2- 172.21.15.57:0/86668660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c10d940 0x7f7b9c115460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009c10 tx=0x7f7b9402a6e0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.143+0000 7f7ba3800640 1 -- 172.21.15.57:0/86668660 shutdown_connections 2024-09-12T03:08:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.143+0000 7f7ba3800640 1 --2- 172.21.15.57:0/86668660 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c10d940 0x7f7b9c115460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.143+0000 7f7ba3800640 1 --2- 172.21.15.57:0/86668660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10eed0 0x7f7b9c10d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.143+0000 7f7ba3800640 1 --2- 172.21.15.57:0/86668660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.144+0000 7f7ba3800640 1 -- 172.21.15.57:0/86668660 >> 172.21.15.57:0/86668660 conn(0x7f7b9c06ce00 msgr2=0x7f7b9c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.144+0000 7f7ba3800640 1 -- 172.21.15.57:0/86668660 shutdown_connections 2024-09-12T03:08:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.144+0000 7f7ba3800640 1 -- 172.21.15.57:0/86668660 wait complete. 2024-09-12T03:08:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.151+0000 7f7ba3800640 1 Processor -- start 2024-09-12T03:08:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.152+0000 7f7ba3800640 1 -- start start 2024-09-12T03:08:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 0x7f7b9c1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 0x7f7b9c1abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1184c0 con 0x7f7b9c1aba20 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c118340 con 0x7f7b9c10e520 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.153+0000 7f7ba3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c118640 con 0x7f7b9c10d940 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.154+0000 7f7b9be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.154+0000 7f7ba1800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 0x7f7b9c1abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.154+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 0x7f7b9c1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.155+0000 7f7b9be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c1a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:43500/0 (socket says 172.21.15.57:43500) 2024-09-12T03:08:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.155+0000 7f7b9be00640 1 -- 172.21.15.57:0/4215647611 learned_addr learned my addr 172.21.15.57:0/4215647611 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba0e00640 1 -- 172.21.15.57:0/4215647611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 msgr2=0x7f7b9c1a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c1a76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba0e00640 1 -- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 msgr2=0x7f7b9c1abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 0x7f7b9c1abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba0e00640 1 -- 172.21.15.57:0/4215647611 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1ac510 con 0x7f7b9c10d940 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.156+0000 7f7ba1800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 0x7f7b9c1abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.157+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 0x7f7b9c1a7160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.158+0000 7f7b99600640 1 -- 172.21.15.57:0/4215647611 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c009c30 con 0x7f7b9c10d940 2024-09-12T03:08:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.158+0000 7f7b99600640 1 -- 172.21.15.57:0/4215647611 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c009dd0 con 0x7f7b9c10d940 2024-09-12T03:08:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.158+0000 7f7b99600640 1 -- 172.21.15.57:0/4215647611 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c010460 con 0x7f7b9c10d940 2024-09-12T03:08:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.160+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1ac800 con 0x7f7b9c10d940 2024-09-12T03:08:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.160+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1b3f70 con 0x7f7b9c10d940 2024-09-12T03:08:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.167+0000 7f7b99600640 1 -- 172.21.15.57:0/4215647611 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f7b8c003f50 con 0x7f7b9c10d940 2024-09-12T03:08:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.170+0000 7f7b99600640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 0x7f7b800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.170+0000 7f7b99600640 1 -- 172.21.15.57:0/4215647611 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f7b8c0cf380 con 0x7f7b9c10d940 2024-09-12T03:08:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.172+0000 7f7ba666d740 1 Processor -- start 2024-09-12T03:08:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.172+0000 7f7ba666d740 1 -- start start 2024-09-12T03:08:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.173+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.173+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c683019b00 0x55c683023e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.173+0000 7f7b9be00640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 0x7f7b800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.173+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683026790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.174+0000 7f7ba666d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c682f7f100 con 0x55c6830191e0 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.174+0000 7f7ba666d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c682f271b0 con 0x55c6830243a0 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.174+0000 7f7ba666d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c682f7b0c0 con 0x55c683019b00 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45340/0 (socket says 172.21.15.57:45340) 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 -- 172.21.15.57:0/3925581064 learned_addr learned my addr 172.21.15.57:0/3925581064 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba1800640 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683026790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 -- 172.21.15.57:0/3925581064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c683019b00 msgr2=0x55c683023e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c683019b00 0x55c683023e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 -- 172.21.15.57:0/3925581064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 msgr2=0x55c683026790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683026790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.175+0000 7f7ba0e00640 1 -- 172.21.15.57:0/3925581064 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c682c92210 con 0x55c6830191e0 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.176+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c09b480 tx=0x7f7b8c09b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c6f0536293a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.176+0000 7f7b92000640 1 -- 172.21.15.57:0/3925581064 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c09b4f0 con 0x55c6830191e0 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.177+0000 7f7b92000640 1 -- 172.21.15.57:0/3925581064 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c09f8a0 con 0x55c6830191e0 2024-09-12T03:08:00.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.177+0000 7f7b92000640 1 -- 172.21.15.57:0/3925581064 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0a3cd0 con 0x55c6830191e0 2024-09-12T03:08:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 -- 172.21.15.57:0/3925581064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 msgr2=0x55c6830195c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c09b480 tx=0x7f7b8c09b0f0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7b9be00640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 0x7f7b800a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55c6830165b0 tx=0x7f7b8800c040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 -- 172.21.15.57:0/3925581064 shutdown_connections 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683026790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c683019b00 0x55c683023e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 --2- 172.21.15.57:0/3925581064 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c6830195c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.178+0000 7f7ba666d740 1 -- 172.21.15.57:0/3925581064 >> 172.21.15.57:0/3925581064 conn(0x55c683015750 msgr2=0x55c683015b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.179+0000 7f7ba666d740 1 -- 172.21.15.57:0/3925581064 shutdown_connections 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.179+0000 7f7ba666d740 1 -- 172.21.15.57:0/3925581064 wait complete. 2024-09-12T03:08:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.180+0000 7f7ba666d740 1 -- start start 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.180+0000 7f7ba666d740 1 Processor -- start 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c683021320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c683019b00 0x55c683021860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 0x55c683021da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c683026f70 con 0x55c6830191e0 2024-09-12T03:08:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c682f08090 con 0x55c683019b00 2024-09-12T03:08:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c6830270f0 con 0x55c6830243a0 2024-09-12T03:08:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba666d740 10 client.0 _ll_register_callbacks cb 0x55c682f5cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c683021320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c683021320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45354/0 (socket says 172.21.15.57:45354) 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba0e00640 1 -- 172.21.15.57:0/2957523283 learned_addr learned my addr 172.21.15.57:0/2957523283 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.181+0000 7f7ba1800640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 0x55c683021da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 -- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c683019b00 msgr2=0x55c683021860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c683019b00 0x55c683021860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 -- 172.21.15.57:0/2957523283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 msgr2=0x55c683021320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c683021320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 -- 172.21.15.57:0/2957523283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c682d5e620 con 0x55c6830243a0 2024-09-12T03:08:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.182+0000 7f7ba1800640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 0x55c683021da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b94034040 tx=0x7f7b94005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bee1bf44b234503 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.183+0000 7f7b86a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.183+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94042070 con 0x55c6830243a0 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.183+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c09f8a0 con 0x55c6830243a0 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.183+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f7b94003d80 con 0x55c6830243a0 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.184+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b9404e050 con 0x55c6830243a0 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.184+0000 7f7ba666d740 10 client.25429 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.184+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c683032bf0 con 0x55c6830243a0 2024-09-12T03:08:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.184+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b8c09f8a0 con 0x55c6830243a0 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b9403c040 con 0x55c6830243a0 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 10 client.25429 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 10 client.25429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 20 client.25429 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c683034120 0x55c683036550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 4 client.25429 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c682f8c830 con 0x55c683034120 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.185+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c683034120 0x55c683036550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.186+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c683034120 0x55c683036550 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f83c7b02b4c1aa server_cookie=a9a2c6fb54e2b0bf in_seq=0 out_seq=0 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.186+0000 7f7b86a00640 10 client.25429 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7b86a00640 1 -- 172.21.15.57:0/2957523283 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c682f8c830 con 0x55c683034120 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7b86a00640 10 client.25429 handle_command_reply: tid=0 2024-09-12T03:08:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 client.25429 shutdown 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 msgr2=0x55c683021da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 0x55c683021da0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b94034040 tx=0x7f7b94005e50 comp rx=0 tx=0).stop 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 shutdown_connections 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c683034120 0x55c683036550 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830243a0 0x55c683021da0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c683019b00 0x55c683021860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 --2- 172.21.15.57:0/2957523283 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c6830191e0 0x55c683021320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7b86a00640 0 client.25429 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.187+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 >> 172.21.15.57:0/2957523283 conn(0x55c683015750 msgr2=0x55c68301b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.188+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 shutdown_connections 2024-09-12T03:08:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.188+0000 7f7ba666d740 1 -- 172.21.15.57:0/2957523283 wait complete. 2024-09-12T03:08:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.188+0000 7f7ba666d740 20 client.25429 trim_cache size 0 max 16384 2024-09-12T03:08:00.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.220+0000 7f7ba666d740 1 Processor -- start 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.220+0000 7f7ba666d740 1 -- start start 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c68301bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c68303be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1c0860 con 0x55c683019b00 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1be130 con 0x55c6830243a0 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba666d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1bbf80 con 0x55c6830191e0 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7b9be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7b9be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45368/0 (socket says 172.21.15.57:45368) 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7b9be00640 1 -- 172.21.15.57:0/226956507 learned_addr learned my addr 172.21.15.57:0/226956507 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.221+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c68301bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 -- 172.21.15.57:0/226956507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 msgr2=0x55c68301bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c68301bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 -- 172.21.15.57:0/226956507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 msgr2=0x55c68303be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c68303be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 -- 172.21.15.57:0/226956507 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c683019010 con 0x55c683019b00 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.222+0000 7f7b9be00640 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fcc4dd8bce94a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.223+0000 7f7b91600640 1 -- 172.21.15.57:0/226956507 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88026070 con 0x55c683019b00 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.223+0000 7f7b91600640 1 -- 172.21.15.57:0/226956507 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88016db0 con 0x55c683019b00 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.223+0000 7f7b91600640 1 -- 172.21.15.57:0/226956507 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88014920 con 0x55c683019b00 2024-09-12T03:08:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 -- 172.21.15.57:0/226956507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 msgr2=0x55c683023180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88004730 comp rx=0 tx=0).stop 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 -- 172.21.15.57:0/226956507 shutdown_connections 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c68303be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c683023180 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 --2- 172.21.15.57:0/226956507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c68301bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 -- 172.21.15.57:0/226956507 >> 172.21.15.57:0/226956507 conn(0x55c683015750 msgr2=0x55c683022a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 -- 172.21.15.57:0/226956507 shutdown_connections 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.224+0000 7f7ba666d740 1 -- 172.21.15.57:0/226956507 wait complete. 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.225+0000 7f7ba666d740 1 -- start start 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.225+0000 7f7ba666d740 1 Processor -- start 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c6830284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683028a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1bc5f0 con 0x55c683019b00 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1bda60 con 0x55c6830243a0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1bccc0 con 0x55c6830191e0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba666d740 10 client.0 _ll_register_callbacks cb 0x7f7b94009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51454/0 (socket says 172.21.15.57:51454) 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba0e00640 1 -- 172.21.15.57:0/1804721618 learned_addr learned my addr 172.21.15.57:0/1804721618 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c6830284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba1800640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683028a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 -- 172.21.15.57:0/1804721618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 msgr2=0x55c683027fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 -- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 msgr2=0x55c683028a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683028a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7b9be00640 1 -- 172.21.15.57:0/1804721618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c683010d60 con 0x55c683019b00 2024-09-12T03:08:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.226+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.227+0000 7f7b9be00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c6830284e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c683019fb0 tx=0x7f7b88021a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36431b170643549b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.227+0000 7f7b86a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.227+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88026040 con 0x55c683019b00 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.227+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f7b880145a0 con 0x55c683019b00 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.227+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b88016db0 con 0x55c683019b00 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.228+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b88038050 con 0x55c683019b00 2024-09-12T03:08:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.228+0000 7f7ba666d740 10 client.15366 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.228+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c68309ee70 con 0x55c683019b00 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.228+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b88016db0 con 0x55c683019b00 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b8802a560 con 0x55c683019b00 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 10 client.15366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 20 client.15366 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c68309f590 0x55c6830a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 4 client.15366 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c682f86b80 con 0x55c68309f590 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.229+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c68309f590 0x55c6830a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.230+0000 7f7ba0e00640 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c68309f590 0x55c6830a19b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267cbcc4f4a80ab9 server_cookie=c924832bd3b08be2 in_seq=0 out_seq=0 2024-09-12T03:08:00.233 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.230+0000 7f7b86a00640 10 client.15366 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.238+0000 7f7b86a00640 1 -- 172.21.15.57:0/1804721618 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c682f86b80 con 0x55c68309f590 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7b86a00640 10 client.15366 handle_command_reply: tid=0 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 client.15366 shutdown 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 msgr2=0x55c6830284e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c6830284e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c683019fb0 tx=0x7f7b88021a70 comp rx=0 tx=0).stop 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 shutdown_connections 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c68309f590 0x55c6830a19b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c6830243a0 0x55c683028a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c683019b00 0x55c6830284e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 --2- 172.21.15.57:0/1804721618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6830191e0 0x55c683027fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7b86a00640 0 client.15366 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 >> 172.21.15.57:0/1804721618 conn(0x55c683015750 msgr2=0x55c683021ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 shutdown_connections 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 1 -- 172.21.15.57:0/1804721618 wait complete. 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.239+0000 7f7ba666d740 20 client.15366 trim_cache size 0 max 16384 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:08:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 msgr2=0x7f7b800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 0x7f7b800a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55c6830165b0 tx=0x7f7b8800c040 comp rx=0 tx=0).stop 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 msgr2=0x7f7b9c1a7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 0x7f7b9c1a7160 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005fd0 tx=0x7f7b8c00dde0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 shutdown_connections 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b800a2e20 0x7f7b800a52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b9c1aba20 0x7f7b9c1abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b9c10e520 0x7f7b9c1a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 --2- 172.21.15.57:0/4215647611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b9c10d940 0x7f7b9c1a7160 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.240+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 >> 172.21.15.57:0/4215647611 conn(0x7f7b9c06ce00 msgr2=0x7f7b9c071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.241+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 shutdown_connections 2024-09-12T03:08:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:00.241+0000 7f7ba3800640 1 -- 172.21.15.57:0/4215647611 wait complete. 2024-09-12T03:08:01.377 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:01 smithi057 ceph-mon[30361]: pgmap v65: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 215 MiB/s wr, 845 op/s 2024-09-12T03:08:01.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:01 smithi204 ceph-mon[35013]: pgmap v65: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 215 MiB/s wr, 845 op/s 2024-09-12T03:08:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:01 smithi181 ceph-mon[35320]: pgmap v65: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 215 MiB/s wr, 845 op/s 2024-09-12T03:08:03.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:03 smithi057 ceph-mon[30361]: pgmap v66: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 305 MiB/s wr, 1.18k op/s 2024-09-12T03:08:03.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:03 smithi204 ceph-mon[35013]: pgmap v66: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 305 MiB/s wr, 1.18k op/s 2024-09-12T03:08:03.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:03 smithi181 ceph-mon[35320]: pgmap v66: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 305 MiB/s wr, 1.18k op/s 2024-09-12T03:08:04.383 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:04 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:04.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:04 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:04.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:04 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:08:05.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:05 smithi057 ceph-mon[30361]: pgmap v67: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 906 op/s 2024-09-12T03:08:05.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:05 smithi204 ceph-mon[35013]: pgmap v67: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 906 op/s 2024-09-12T03:08:05.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:05 smithi181 ceph-mon[35320]: pgmap v67: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 240 MiB/s wr, 906 op/s 2024-09-12T03:08:06.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:06 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:06.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:06 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:06.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:06 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:08:06.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:06 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:06.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:06 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:06.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:06 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:08:06.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:06 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:08:06.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:06 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:08:06.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:06 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:08:07.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:07 smithi057 ceph-mon[30361]: pgmap v68: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 211 MiB/s wr, 784 op/s 2024-09-12T03:08:07.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:07 smithi204 ceph-mon[35013]: pgmap v68: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 211 MiB/s wr, 784 op/s 2024-09-12T03:08:07.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:07 smithi181 ceph-mon[35320]: pgmap v68: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 211 MiB/s wr, 784 op/s 2024-09-12T03:08:09.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:09 smithi057 ceph-mon[30361]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 309 MiB/s wr, 1.31k op/s 2024-09-12T03:08:09.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:09 smithi204 ceph-mon[35013]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 309 MiB/s wr, 1.31k op/s 2024-09-12T03:08:09.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:09 smithi181 ceph-mon[35320]: pgmap v69: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 309 MiB/s wr, 1.31k op/s 2024-09-12T03:08:11.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:11 smithi204 ceph-mon[35013]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 201 MiB/s wr, 929 op/s 2024-09-12T03:08:11.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:11 smithi181 ceph-mon[35320]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 201 MiB/s wr, 929 op/s 2024-09-12T03:08:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:11 smithi057 ceph-mon[30361]: pgmap v70: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 201 MiB/s wr, 929 op/s 2024-09-12T03:08:13.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:13 smithi204 ceph-mon[35013]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 274 MiB/s wr, 1.50k op/s 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 -- 172.21.15.57:0/1955624222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 msgr2=0x7f5ec4103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 --2- 172.21.15.57:0/1955624222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec4103060 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009900 tx=0x7f5eb402a480 comp rx=0 tx=0).stop 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 -- 172.21.15.57:0/1955624222 shutdown_connections 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 --2- 172.21.15.57:0/1955624222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec410c0d0 0x7f5ec410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 --2- 172.21.15.57:0/1955624222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec41035a0 0x7f5ec410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 --2- 172.21.15.57:0/1955624222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec4103060 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.446+0000 7f5ecac00640 1 -- 172.21.15.57:0/1955624222 >> 172.21.15.57:0/1955624222 conn(0x7f5ec4077990 msgr2=0x7f5ec40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.447+0000 7f5ecac00640 1 -- 172.21.15.57:0/1955624222 shutdown_connections 2024-09-12T03:08:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.447+0000 7f5ecac00640 1 -- 172.21.15.57:0/1955624222 wait complete. 2024-09-12T03:08:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.448+0000 7f5ecac00640 1 Processor -- start 2024-09-12T03:08:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 -- start start 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec410b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec410c0d0 0x7f5ec410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410f420 con 0x7f5ec410c0d0 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410f2a0 con 0x7f5ec4102c80 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ecac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec410f5a0 con 0x7f5ec41035a0 2024-09-12T03:08:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.449+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55812/0 (socket says 172.21.15.57:55812) 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 -- 172.21.15.57:0/3915054375 learned_addr learned my addr 172.21.15.57:0/3915054375 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec410b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec410c0d0 0x7f5ec410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 -- 172.21.15.57:0/3915054375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 msgr2=0x7f5ec410b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec410b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 -- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec410c0d0 msgr2=0x7f5ec410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec410c0d0 0x7f5ec410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 -- 172.21.15.57:0/3915054375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec41094f0 con 0x7f5ec41035a0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.450+0000 7f5ec3400640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c970 tx=0x7f5eb000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.451+0000 7f5ec0c00640 1 -- 172.21.15.57:0/3915054375 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0007bb0 con 0x7f5ec41035a0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.451+0000 7f5ec0c00640 1 -- 172.21.15.57:0/3915054375 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb0007d50 con 0x7f5ec41035a0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.451+0000 7f5ec0c00640 1 -- 172.21.15.57:0/3915054375 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0018820 con 0x7f5ec41035a0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.451+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec4109780 con 0x7f5ec41035a0 2024-09-12T03:08:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.451+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec4109c80 con 0x7f5ec41035a0 2024-09-12T03:08:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:13 smithi181 ceph-mon[35320]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 274 MiB/s wr, 1.50k op/s 2024-09-12T03:08:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.457+0000 7f5ec0c00640 1 -- 172.21.15.57:0/3915054375 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f5eb00189c0 con 0x7f5ec41035a0 2024-09-12T03:08:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.458+0000 7f5ecdc23740 1 Processor -- start 2024-09-12T03:08:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 -- start start 2024-09-12T03:08:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x562634780480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x56263479be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5626346e6100 con 0x5626347800a0 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56263468e1b0 con 0x56263479c3b0 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5626346e20c0 con 0x5626347809c0 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.459+0000 7f5ec0c00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 0x7f5ea80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec0c00640 1 -- 172.21.15.57:0/3915054375 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5eb00cf8f0 con 0x7f5ec41035a0 2024-09-12T03:08:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:43546/0 (socket says 172.21.15.57:43546) 2024-09-12T03:08:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 -- 172.21.15.57:0/2825608776 learned_addr learned my addr 172.21.15.57:0/2825608776 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 -- 172.21.15.57:0/2825608776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 msgr2=0x56263479be70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:08:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x56263479be70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 -- 172.21.15.57:0/2825608776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 msgr2=0x562634780480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x562634780480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 -- 172.21.15.57:0/2825608776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626343f9210 con 0x56263479c3b0 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 0x7f5ea80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.460+0000 7f5ec3400640 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00a41a0 tx=0x7f5eb009b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9278bc101f24ee69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.461+0000 7f5eb9200640 1 -- 172.21.15.57:0/2825608776 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb009c160 con 0x56263479c3b0 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.461+0000 7f5eb9200640 1 -- 172.21.15.57:0/2825608776 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb009ba60 con 0x56263479c3b0 2024-09-12T03:08:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.461+0000 7f5eb9200640 1 -- 172.21.15.57:0/2825608776 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00ccf10 con 0x56263479c3b0 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.462+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2825608776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 msgr2=0x56263479e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.462+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00a41a0 tx=0x7f5eb009b660 comp rx=0 tx=0).stop 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2825608776 shutdown_connections 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x56263479e7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x56263479be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2825608776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x562634780480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2825608776 >> 172.21.15.57:0/2825608776 conn(0x56263477c750 msgr2=0x56263477cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 0x7f5ea80a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009270 tx=0x7f5ebc0091e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2825608776 shutdown_connections 2024-09-12T03:08:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.463+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2825608776 wait complete. 2024-09-12T03:08:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.464+0000 7f5ecdc23740 1 -- start start 2024-09-12T03:08:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.464+0000 7f5ecdc23740 1 Processor -- start 2024-09-12T03:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 0x562634799390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 0x5626347998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x562634799e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 0x562634799390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 0x5626347998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 0x562634799390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55840/0 (socket says 172.21.15.57:55840) 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec3e00640 1 -- 172.21.15.57:0/2164918994 learned_addr learned my addr 172.21.15.57:0/2164918994 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec3400640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 0x5626347998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36606/0 (socket says 172.21.15.57:36606) 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x562634799e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.465+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56263479ef20 con 0x5626347809c0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56263466f090 con 0x56263479c3b0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56263479f0a0 con 0x5626347800a0 2024-09-12T03:08:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ecdc23740 10 client.0 _ll_register_callbacks cb 0x5626346c3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 -- 172.21.15.57:0/2164918994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 msgr2=0x562634799390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 0x562634799390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 -- 172.21.15.57:0/2164918994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 msgr2=0x5626347998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 0x5626347998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626344c5620 con 0x56263479c3b0 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb009ba60 con 0x56263479c3b0 2024-09-12T03:08:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x562634799e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193c9ef150dbe1d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.466+0000 7f5ead600640 10 client.25427 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ead600640 1 -- 172.21.15.57:0/2164918994 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc026420 con 0x56263479c3b0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ead600640 1 -- 172.21.15.57:0/2164918994 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5ebc0265c0 con 0x56263479c3b0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ead600640 1 -- 172.21.15.57:0/2164918994 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc0267c0 con 0x56263479c3b0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ecdc23740 10 client.25427 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5626347aac00 con 0x56263479c3b0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ead600640 1 -- 172.21.15.57:0/2164918994 <== mon.2 v2:172.21.15.181:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5ebc026960 con 0x56263479c3b0 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.467+0000 7f5ecdc23740 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ecdc23740 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ecdc23740 20 client.25427 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5626347ac000 0x5626347ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ecdc23740 4 client.25427 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626346f3830 con 0x5626347ac000 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.468+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5626347ac000 0x5626347ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.469+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5626347ac000 0x5626347ae470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c28410a9d04dde server_cookie=d7bf4e4cc261770d in_seq=0 out_seq=0 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.469+0000 7f5ead600640 10 client.25427 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.469+0000 7f5ead600640 1 -- 172.21.15.57:0/2164918994 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626346f3830 con 0x5626347ac000 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.469+0000 7f5ead600640 10 client.25427 handle_command_reply: tid=0 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 client.25427 shutdown 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 msgr2=0x562634799e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x562634799e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc009500 comp rx=0 tx=0).stop 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 shutdown_connections 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5626347ac000 0x5626347ae470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x562634799e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347809c0 0x5626347998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/2164918994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347800a0 0x562634799390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ead600640 0 client.25427 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 >> 172.21.15.57:0/2164918994 conn(0x56263477c750 msgr2=0x5626347824b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 shutdown_connections 2024-09-12T03:08:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.470+0000 7f5ecdc23740 1 -- 172.21.15.57:0/2164918994 wait complete. 2024-09-12T03:08:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.471+0000 7f5ecdc23740 20 client.25427 trim_cache size 0 max 16384 2024-09-12T03:08:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 Processor -- start 2024-09-12T03:08:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 -- start start 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41b77a0 con 0x5626347800a0 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41b7e70 con 0x56263479c3b0 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41b5740 con 0x5626347809c0 2024-09-12T03:08:13.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55856/0 (socket says 172.21.15.57:55856) 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.497+0000 7f5ec3400640 1 -- 172.21.15.57:0/420415522 learned_addr learned my addr 172.21.15.57:0/420415522 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 -- 172.21.15.57:0/420415522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 msgr2=0x5626347965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 -- 172.21.15.57:0/420415522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 msgr2=0x5626347b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 -- 172.21.15.57:0/420415522 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626347aead0 con 0x5626347809c0 2024-09-12T03:08:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5ec3400640 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0018e20 tx=0x7f5eb0051160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3722b340b1c2cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.498+0000 7f5eafe00640 1 -- 172.21.15.57:0/420415522 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00b8070 con 0x5626347809c0 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.499+0000 7f5eafe00640 1 -- 172.21.15.57:0/420415522 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb000fac0 con 0x5626347809c0 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5eafe00640 1 -- 172.21.15.57:0/420415522 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00cd120 con 0x5626347809c0 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- 172.21.15.57:0/420415522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 msgr2=0x5626347b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0018e20 tx=0x7f5eb0051160 comp rx=0 tx=0).stop 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- 172.21.15.57:0/420415522 shutdown_connections 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347b42a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/420415522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- 172.21.15.57:0/420415522 >> 172.21.15.57:0/420415522 conn(0x56263477c750 msgr2=0x56263479b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- 172.21.15.57:0/420415522 shutdown_connections 2024-09-12T03:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- 172.21.15.57:0/420415522 wait complete. 2024-09-12T03:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.500+0000 7f5ecdc23740 1 -- start start 2024-09-12T03:08:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.501+0000 7f5ecdc23740 1 Processor -- start 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.501+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.501+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ecdc23740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec411bed0 con 0x5626347800a0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41b5070 con 0x56263479c3b0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ecdc23740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec41b42f0 con 0x5626347809c0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ecdc23740 10 client.0 _ll_register_callbacks cb 0x7f5ebc0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ec3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ec3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:36622/0 (socket says 172.21.15.57:36622) 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ec3e00640 1 -- 172.21.15.57:0/4066180152 learned_addr learned my addr 172.21.15.57:0/4066180152 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.502+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 -- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 msgr2=0x5626347a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 -- 172.21.15.57:0/4066180152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 msgr2=0x5626347a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 -- 172.21.15.57:0/4066180152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562634815f80 con 0x5626347809c0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec8c00640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ec3400640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347a07f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56263479a480 tx=0x7f5eb00cd1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1760caa349ac9765 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.503+0000 7f5ead600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00b8070 con 0x5626347809c0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb000fac0 con 0x5626347809c0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f5eb00b9d90 con 0x5626347809c0 2024-09-12T03:08:13.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5eb009c160 con 0x5626347809c0 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ecdc23740 10 client.25450 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562634817250 con 0x5626347809c0 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.504+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb000fac0 con 0x5626347809c0 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5eb009afd0 con 0x5626347809c0 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 10 client.25450 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:13.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 20 client.25450 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562634817480 0x562634819870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 4 client.25450 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.505+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626346edb80 con 0x562634817480 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.506+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562634817480 0x562634819870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.507+0000 7f5ec3e00640 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562634817480 0x562634819870 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef14f126d37b43d server_cookie=c96b8d13424adf96 in_seq=0 out_seq=0 2024-09-12T03:08:13.508 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.507+0000 7f5ead600640 10 client.25450 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.521+0000 7f5ead600640 1 -- 172.21.15.57:0/4066180152 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626346edb80 con 0x562634817480 2024-09-12T03:08:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.521+0000 7f5ead600640 10 client.25450 handle_command_reply: tid=0 2024-09-12T03:08:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.521+0000 7f5ecdc23740 1 client.25450 shutdown 2024-09-12T03:08:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 msgr2=0x5626347a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347a07f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56263479a480 tx=0x7f5eb00cd1c0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 shutdown_connections 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562634817480 0x562634819870 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56263479c3b0 0x5626347a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626347809c0 0x5626347a07f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 --2- 172.21.15.57:0/4066180152 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5626347800a0 0x5626347a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 >> 172.21.15.57:0/4066180152 conn(0x56263477c750 msgr2=0x562634799b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.522+0000 7f5ead600640 0 client.25450 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.523+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 shutdown_connections 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.523+0000 7f5ecdc23740 1 -- 172.21.15.57:0/4066180152 wait complete. 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.523+0000 7f5ecdc23740 20 client.25450 trim_cache size 0 max 16384 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:08:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.524+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 msgr2=0x7f5ea80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.524+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 0x7f5ea80a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009270 tx=0x7f5ebc0091e0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.524+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 msgr2=0x7f5ec4073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.524+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000c970 tx=0x7f5eb000ce30 comp rx=0 tx=0).stop 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 shutdown_connections 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5ea80a2e20 0x7f5ea80a52e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ec410c0d0 0x7f5ec410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec41035a0 0x7f5ec4073d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 --2- 172.21.15.57:0/3915054375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec4102c80 0x7f5ec410b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 >> 172.21.15.57:0/3915054375 conn(0x7f5ec4077990 msgr2=0x7f5ec40fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 shutdown_connections 2024-09-12T03:08:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:13.525+0000 7f5ecac00640 1 -- 172.21.15.57:0/3915054375 wait complete. 2024-09-12T03:08:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:13 smithi057 ceph-mon[30361]: pgmap v71: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 274 MiB/s wr, 1.50k op/s 2024-09-12T03:08:15.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:15 smithi204 ceph-mon[35013]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.24k op/s 2024-09-12T03:08:15.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:15 smithi181 ceph-mon[35320]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.24k op/s 2024-09-12T03:08:15.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:15 smithi057 ceph-mon[30361]: pgmap v72: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.24k op/s 2024-09-12T03:08:17.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:17 smithi181 ceph-mon[35320]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 181 MiB/s wr, 1.18k op/s 2024-09-12T03:08:17.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:17 smithi057 ceph-mon[30361]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 181 MiB/s wr, 1.18k op/s 2024-09-12T03:08:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:17 smithi204 ceph-mon[35013]: pgmap v73: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 181 MiB/s wr, 1.18k op/s 2024-09-12T03:08:19.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:19 smithi181 ceph-mon[35320]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 200 MiB/s wr, 1.66k op/s 2024-09-12T03:08:19.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:19 smithi057 ceph-mon[30361]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 200 MiB/s wr, 1.66k op/s 2024-09-12T03:08:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:19 smithi204 ceph-mon[35013]: pgmap v74: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 200 MiB/s wr, 1.66k op/s 2024-09-12T03:08:21.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:21 smithi204 ceph-mon[35013]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 102 MiB/s wr, 1.13k op/s 2024-09-12T03:08:21.452 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:21 smithi181 ceph-mon[35320]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 102 MiB/s wr, 1.13k op/s 2024-09-12T03:08:21.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:21 smithi057 ceph-mon[30361]: pgmap v75: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 102 MiB/s wr, 1.13k op/s 2024-09-12T03:08:23.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:23 smithi181 ceph-mon[35320]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 190 MiB/s wr, 1.98k op/s 2024-09-12T03:08:23.633 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:23 smithi057 ceph-mon[30361]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 190 MiB/s wr, 1.98k op/s 2024-09-12T03:08:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:23 smithi204 ceph-mon[35013]: pgmap v76: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 190 MiB/s wr, 1.98k op/s 2024-09-12T03:08:25.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:25 smithi057 ceph-mon[30361]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 155 MiB/s wr, 1.78k op/s 2024-09-12T03:08:25.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:25 smithi204 ceph-mon[35013]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 155 MiB/s wr, 1.78k op/s 2024-09-12T03:08:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:25 smithi181 ceph-mon[35320]: pgmap v77: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 155 MiB/s wr, 1.78k op/s 2024-09-12T03:08:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.875+0000 7efde6e00640 1 -- 172.21.15.57:0/533735254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde010d940 msgr2=0x7efde0115460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.875+0000 7efddea00640 1 -- 172.21.15.57:0/533735254 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0030980 con 0x7efde010d940 2024-09-12T03:08:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.875+0000 7efde6e00640 1 --2- 172.21.15.57:0/533735254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde010d940 0x7efde0115460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009890 tx=0x7efdd002a360 comp rx=0 tx=0).stop 2024-09-12T03:08:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.878+0000 7efde6e00640 1 -- 172.21.15.57:0/533735254 shutdown_connections 2024-09-12T03:08:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.878+0000 7efde6e00640 1 --2- 172.21.15.57:0/533735254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde010d940 0x7efde0115460 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.878+0000 7efde6e00640 1 --2- 172.21.15.57:0/533735254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010eed0 0x7efde010d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.878+0000 7efde6e00640 1 --2- 172.21.15.57:0/533735254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010e520 0x7efde010e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.878+0000 7efde6e00640 1 -- 172.21.15.57:0/533735254 >> 172.21.15.57:0/533735254 conn(0x7efde006ce00 msgr2=0x7efde006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.886+0000 7efde6e00640 1 -- 172.21.15.57:0/533735254 shutdown_connections 2024-09-12T03:08:26.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.886+0000 7efde6e00640 1 -- 172.21.15.57:0/533735254 wait complete. 2024-09-12T03:08:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.904+0000 7efde6e00640 1 Processor -- start 2024-09-12T03:08:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.906+0000 7efde6e00640 1 -- start start 2024-09-12T03:08:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.907+0000 7efde6e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.907+0000 7efde6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 0x7efde0113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.909+0000 7efddfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.909+0000 7efddfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:57998/0 (socket says 172.21.15.57:57998) 2024-09-12T03:08:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efddfe00640 1 -- 172.21.15.57:0/1594825959 learned_addr learned my addr 172.21.15.57:0/1594825959 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efddf400640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 0x7efde0113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde0114640 0x7efde01b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efde0117b10 con 0x7efde0114640 2024-09-12T03:08:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efde0117990 con 0x7efde010d940 2024-09-12T03:08:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.911+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efde0117c90 con 0x7efde010e520 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.912+0000 7efddf400640 1 -- 172.21.15.57:0/1594825959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 msgr2=0x7efde0113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.912+0000 7efddf400640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.912+0000 7efddf400640 1 -- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde0114640 msgr2=0x7efde01b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.912+0000 7efddf400640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde0114640 0x7efde01b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.912+0000 7efddf400640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde01b35f0 con 0x7efde010e520 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.913+0000 7efddf400640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 0x7efde0113fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efdcc005fd0 tx=0x7efdcc00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.915+0000 7efddfe00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.915+0000 7efddcc00640 1 -- 172.21.15.57:0/1594825959 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc015030 con 0x7efde010e520 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.915+0000 7efddcc00640 1 -- 172.21.15.57:0/1594825959 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc011440 con 0x7efde010e520 2024-09-12T03:08:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.915+0000 7efddcc00640 1 -- 172.21.15.57:0/1594825959 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc015030 con 0x7efde010e520 2024-09-12T03:08:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.919+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde01b38e0 con 0x7efde010e520 2024-09-12T03:08:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.920+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01b3de0 con 0x7efde010e520 2024-09-12T03:08:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.935+0000 7efde9cb2740 1 Processor -- start 2024-09-12T03:08:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.937+0000 7efde9cb2740 1 -- start start 2024-09-12T03:08:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.937+0000 7efddcc00640 1 -- 172.21.15.57:0/1594825959 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7efdcc018030 con 0x7efde010e520 2024-09-12T03:08:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.939+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.941+0000 7efddcc00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 0x7efdc40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b313de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3140790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde9cb2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5625b3099100 con 0x5625b313e3a0 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde9cb2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625b30411b0 con 0x5625b31331e0 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde9cb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5625b30950c0 con 0x5625b3133b00 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:58014/0 (socket says 172.21.15.57:58014) 2024-09-12T03:08:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.942+0000 7efde4e00640 1 -- 172.21.15.57:0/1480375805 learned_addr learned my addr 172.21.15.57:0/1480375805 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.943+0000 7efde4e00640 1 -- 172.21.15.57:0/1480375805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 msgr2=0x5625b313de60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:08:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.943+0000 7efde4e00640 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b313de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.943+0000 7efde4e00640 1 -- 172.21.15.57:0/1480375805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 msgr2=0x5625b3140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.943+0000 7efde4e00640 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.943+0000 7efde4e00640 1 -- 172.21.15.57:0/1480375805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b2dac210 con 0x5625b31331e0 2024-09-12T03:08:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.944+0000 7efddfe00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 0x7efdc40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.946+0000 7efddcc00640 1 -- 172.21.15.57:0/1594825959 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7efdcc016070 con 0x7efde010e520 2024-09-12T03:08:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.947+0000 7efde4e00640 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009490 tx=0x7efdd00094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efeff15323f873a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.948+0000 7efdd5600640 1 -- 172.21.15.57:0/1480375805 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0037030 con 0x5625b31331e0 2024-09-12T03:08:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.948+0000 7efdd5600640 1 -- 172.21.15.57:0/1480375805 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd0038040 con 0x5625b31331e0 2024-09-12T03:08:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.949+0000 7efde9cb2740 1 -- 172.21.15.57:0/1480375805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 msgr2=0x5625b31335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.949+0000 7efdd5600640 1 -- 172.21.15.57:0/1480375805 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0037030 con 0x5625b31331e0 2024-09-12T03:08:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.949+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009490 tx=0x7efdd00094c0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.950+0000 7efde9cb2740 1 -- 172.21.15.57:0/1480375805 shutdown_connections 2024-09-12T03:08:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.950+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.950+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b313de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.950+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1480375805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b31335c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.950+0000 7efde9cb2740 1 -- 172.21.15.57:0/1480375805 >> 172.21.15.57:0/1480375805 conn(0x5625b312f750 msgr2=0x5625b312fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.953+0000 7efddfe00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 0x7efdc40a5730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009fd0 tx=0x7efdd8009560 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.954+0000 7efde9cb2740 1 -- 172.21.15.57:0/1480375805 shutdown_connections 2024-09-12T03:08:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.954+0000 7efde9cb2740 1 -- 172.21.15.57:0/1480375805 wait complete. 2024-09-12T03:08:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.955+0000 7efde9cb2740 1 -- start start 2024-09-12T03:08:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.956+0000 7efde9cb2740 1 Processor -- start 2024-09-12T03:08:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b313b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b3133b00 0x5625b313b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5625b3140fc0 con 0x5625b31331e0 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5625b3022090 con 0x5625b3133b00 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.958+0000 7efde9cb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5625b3141140 con 0x5625b313e3a0 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.959+0000 7efde9cb2740 10 client.0 _ll_register_callbacks cb 0x5625b3076f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.960+0000 7efde4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.960+0000 7efde4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:58648/0 (socket says 172.21.15.57:58648) 2024-09-12T03:08:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.960+0000 7efde4e00640 1 -- 172.21.15.57:0/1278925377 learned_addr learned my addr 172.21.15.57:0/1278925377 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.961+0000 7efddf400640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b3133b00 0x5625b313b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.961+0000 7efddfe00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b313b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efde4e00640 1 -- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b3133b00 msgr2=0x5625b313b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efde4e00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b3133b00 0x5625b313b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efde4e00640 1 -- 172.21.15.57:0/1278925377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 msgr2=0x5625b313b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efde4e00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b313b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efde4e00640 1 -- 172.21.15.57:0/1278925377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b2e78620 con 0x5625b313e3a0 2024-09-12T03:08:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.962+0000 7efddfe00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b313b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.963+0000 7efde4e00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdd002ad10 tx=0x7efdd002ad40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea7834c749f7e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.963+0000 7efdca000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.963+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd004a060 con 0x5625b313e3a0 2024-09-12T03:08:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.965+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7efdd0030850 con 0x5625b313e3a0 2024-09-12T03:08:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.967+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd0038040 con 0x5625b313e3a0 2024-09-12T03:08:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.967+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdd0037050 con 0x5625b313e3a0 2024-09-12T03:08:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.968+0000 7efde9cb2740 10 client.25459 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.969+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625b3140fc0 con 0x5625b313e3a0 2024-09-12T03:08:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.970+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd0038040 con 0x5625b313e3a0 2024-09-12T03:08:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.973+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdd0032440 con 0x5625b313e3a0 2024-09-12T03:08:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.973+0000 7efde9cb2740 10 client.25459 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.974+0000 7efde9cb2740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.974+0000 7efde9cb2740 20 client.25459 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.974+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b315ee50 0x5625b3161240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.974+0000 7efde9cb2740 4 client.25459 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.974+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625b30a6830 con 0x5625b315ee50 2024-09-12T03:08:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.977+0000 7efddfe00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b315ee50 0x5625b3161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.977+0000 7efddfe00640 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b315ee50 0x5625b3161240 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8513d3a19e9dcf server_cookie=33506aeb7bf62282 in_seq=0 out_seq=0 2024-09-12T03:08:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.978+0000 7efdca000640 10 client.25459 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.979+0000 7efdca000640 1 -- 172.21.15.57:0/1278925377 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625b30a6830 con 0x5625b315ee50 2024-09-12T03:08:26.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.979+0000 7efdca000640 10 client.25459 handle_command_reply: tid=0 2024-09-12T03:08:26.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.980+0000 7efde9cb2740 1 client.25459 shutdown 2024-09-12T03:08:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.980+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 msgr2=0x5625b313bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.980+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efdd002ad10 tx=0x7efdd002ad40 comp rx=0 tx=0).stop 2024-09-12T03:08:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 shutdown_connections 2024-09-12T03:08:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b315ee50 0x5625b3161240 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b313e3a0 0x5625b313bd90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b3133b00 0x5625b313b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 --2- 172.21.15.57:0/1278925377 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b313b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efdca000640 0 client.25459 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.981+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 >> 172.21.15.57:0/1278925377 conn(0x5625b312f750 msgr2=0x5625b3135460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.992+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 shutdown_connections 2024-09-12T03:08:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.992+0000 7efde9cb2740 1 -- 172.21.15.57:0/1278925377 wait complete. 2024-09-12T03:08:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:26.995+0000 7efde9cb2740 20 client.25459 trim_cache size 0 max 16384 2024-09-12T03:08:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.071+0000 7efde9cb2740 1 Processor -- start 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.071+0000 7efde9cb2740 1 -- start start 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.076+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b312f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.076+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.076+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3166930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.077+0000 7efde9cb2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00128d0 con 0x5625b313e3a0 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.077+0000 7efde9cb2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00101a0 con 0x5625b31331e0 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.077+0000 7efde9cb2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efdc000dfb0 con 0x5625b3133b00 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.077+0000 7efddf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.077+0000 7efddf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:58662/0 (socket says 172.21.15.57:58662) 2024-09-12T03:08:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 -- 172.21.15.57:0/2758815067 learned_addr learned my addr 172.21.15.57:0/2758815067 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 -- 172.21.15.57:0/2758815067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 msgr2=0x5625b312f1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b312f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 -- 172.21.15.57:0/2758815067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 msgr2=0x5625b3166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.078+0000 7efddf400640 1 -- 172.21.15.57:0/2758815067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b31616a0 con 0x5625b3133b00 2024-09-12T03:08:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.081+0000 7efddf400640 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09bdb0 tx=0x7efdcc0981f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75da0696a2a2ade server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.085+0000 7efdd4c00640 1 -- 172.21.15.57:0/2758815067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0b9070 con 0x5625b3133b00 2024-09-12T03:08:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.085+0000 7efdd4c00640 1 -- 172.21.15.57:0/2758815067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc0a01d0 con 0x5625b3133b00 2024-09-12T03:08:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.085+0000 7efdd4c00640 1 -- 172.21.15.57:0/2758815067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc015070 con 0x5625b3133b00 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.086+0000 7efde9cb2740 1 -- 172.21.15.57:0/2758815067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 msgr2=0x5625b3135350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.086+0000 7efde9cb2740 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efdcc09bdb0 tx=0x7efdcc0981f0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.087+0000 7efde9cb2740 1 -- 172.21.15.57:0/2758815067 shutdown_connections 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.087+0000 7efde9cb2740 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b313e3a0 0x5625b3166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.087+0000 7efde9cb2740 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3135350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.087+0000 7efde9cb2740 1 --2- 172.21.15.57:0/2758815067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b31331e0 0x5625b312f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.087+0000 7efde9cb2740 1 -- 172.21.15.57:0/2758815067 >> 172.21.15.57:0/2758815067 conn(0x5625b312f750 msgr2=0x5625b3134c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.092+0000 7efde9cb2740 1 -- 172.21.15.57:0/2758815067 shutdown_connections 2024-09-12T03:08:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.092+0000 7efde9cb2740 1 -- 172.21.15.57:0/2758815067 wait complete. 2024-09-12T03:08:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.093+0000 7efde9cb2740 1 -- start start 2024-09-12T03:08:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.093+0000 7efde9cb2740 1 Processor -- start 2024-09-12T03:08:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.100+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b31420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.102+0000 7efde9cb2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.102+0000 7efddf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.102+0000 7efddf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:58668/0 (socket says 172.21.15.57:58668) 2024-09-12T03:08:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.102+0000 7efddf400640 1 -- 172.21.15.57:0/569917761 learned_addr learned my addr 172.21.15.57:0/569917761 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.103+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 0x5625b3142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.103+0000 7efde4e00640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 0x5625b3142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.104+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7efdc000e660 con 0x5625b31331e0 2024-09-12T03:08:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.104+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7efdc000fad0 con 0x5625b313e3a0 2024-09-12T03:08:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.104+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efdc000ed30 con 0x5625b3133b00 2024-09-12T03:08:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.104+0000 7efde9cb2740 10 client.0 _ll_register_callbacks cb 0x7efdcc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.105+0000 7efddf400640 1 -- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 msgr2=0x5625b3142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.105+0000 7efddf400640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 0x5625b3142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.105+0000 7efddf400640 1 -- 172.21.15.57:0/569917761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 msgr2=0x5625b31420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.106+0000 7efddf400640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b31420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.106+0000 7efddf400640 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b31c8900 con 0x5625b3133b00 2024-09-12T03:08:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.106+0000 7efde4e00640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 0x5625b3142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.107+0000 7efddf400640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00f010 tx=0x7efdcc0cc410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188797017977c5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.107+0000 7efdca000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.107+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc015050 con 0x5625b3133b00 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.107+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7efdcc0a4800 con 0x5625b3133b00 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.107+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc0a01d0 con 0x5625b3133b00 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.108+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdcc0b9040 con 0x5625b3133b00 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.108+0000 7efde9cb2740 10 client.25465 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.108+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5625b31c9b50 con 0x5625b3133b00 2024-09-12T03:08:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.109+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc0a01d0 con 0x5625b3133b00 2024-09-12T03:08:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.109+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdcc0a1360 con 0x5625b3133b00 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 10 client.25465 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 20 client.25465 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b31ca2d0 0x5625b31cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 4 client.25465 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:08:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.113+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625b312ad80 con 0x5625b31ca2d0 2024-09-12T03:08:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.114+0000 7efddfe00640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b31ca2d0 0x5625b31cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.118+0000 7efddfe00640 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b31ca2d0 0x5625b31cc6f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c599cb74d55319 server_cookie=7fbcc05582fd6622 in_seq=0 out_seq=0 2024-09-12T03:08:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.118+0000 7efdca000640 10 client.25465 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.192+0000 7efdca000640 1 -- 172.21.15.57:0/569917761 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625b312ad80 con 0x5625b31ca2d0 2024-09-12T03:08:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.192+0000 7efdca000640 10 client.25465 handle_command_reply: tid=0 2024-09-12T03:08:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.192+0000 7efde9cb2740 1 client.25465 shutdown 2024-09-12T03:08:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.194+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 msgr2=0x5625b3142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00f010 tx=0x7efdcc0cc410 comp rx=0 tx=0).stop 2024-09-12T03:08:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 shutdown_connections 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5625b31ca2d0 0x5625b31cc6f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5625b313e3a0 0x5625b3142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efdca000640 0 client.25465 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5625b3133b00 0x5625b3142630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 --2- 172.21.15.57:0/569917761 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5625b31331e0 0x5625b31420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.195+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 >> 172.21.15.57:0/569917761 conn(0x5625b312f750 msgr2=0x5625b3135460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.211+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 shutdown_connections 2024-09-12T03:08:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.212+0000 7efde9cb2740 1 -- 172.21.15.57:0/569917761 wait complete. 2024-09-12T03:08:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.214+0000 7efde9cb2740 20 client.25465 trim_cache size 0 max 16384 2024-09-12T03:08:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:08:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:08:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:08:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:08:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.236+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 msgr2=0x7efdc40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.236+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 0x7efdc40a5730 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009fd0 tx=0x7efdd8009560 comp rx=0 tx=0).stop 2024-09-12T03:08:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.237+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 msgr2=0x7efde0113fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.237+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 0x7efde0113fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efdcc005fd0 tx=0x7efdcc00edc0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.237+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 shutdown_connections 2024-09-12T03:08:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.238+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7efdc40a3270 0x7efdc40a5730 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.238+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7efde0114640 0x7efde01b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.238+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde010e520 0x7efde0113fa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.238+0000 7efde6e00640 1 --2- 172.21.15.57:0/1594825959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efde010d940 0x7efde0113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.238+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 >> 172.21.15.57:0/1594825959 conn(0x7efde006ce00 msgr2=0x7efde01091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.241+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 shutdown_connections 2024-09-12T03:08:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:27.242+0000 7efde6e00640 1 -- 172.21.15.57:0/1594825959 wait complete. 2024-09-12T03:08:27.625 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:27 smithi057 ceph-mon[30361]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 144 MiB/s wr, 1.70k op/s 2024-09-12T03:08:27.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:27 smithi204 ceph-mon[35013]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 144 MiB/s wr, 1.70k op/s 2024-09-12T03:08:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:27 smithi181 ceph-mon[35320]: pgmap v78: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 144 MiB/s wr, 1.70k op/s 2024-09-12T03:08:29.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:29 smithi057 ceph-mon[30361]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 217 MiB/s wr, 2.37k op/s 2024-09-12T03:08:29.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:29 smithi204 ceph-mon[35013]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 217 MiB/s wr, 2.37k op/s 2024-09-12T03:08:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:29 smithi181 ceph-mon[35320]: pgmap v79: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 217 MiB/s wr, 2.37k op/s 2024-09-12T03:08:31.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:31 smithi057 ceph-mon[30361]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 198 MiB/s wr, 1.89k op/s 2024-09-12T03:08:31.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:31 smithi204 ceph-mon[35013]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 198 MiB/s wr, 1.89k op/s 2024-09-12T03:08:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:31 smithi181 ceph-mon[35320]: pgmap v80: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 198 MiB/s wr, 1.89k op/s 2024-09-12T03:08:33.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:33 smithi057 ceph-mon[30361]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 257 MiB/s wr, 2.43k op/s 2024-09-12T03:08:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:33 smithi204 ceph-mon[35013]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 257 MiB/s wr, 2.43k op/s 2024-09-12T03:08:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:33 smithi181 ceph-mon[35320]: pgmap v81: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 257 MiB/s wr, 2.43k op/s 2024-09-12T03:08:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:35 smithi057 ceph-mon[30361]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 200 MiB/s wr, 1.87k op/s 2024-09-12T03:08:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:35 smithi204 ceph-mon[35013]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 200 MiB/s wr, 1.87k op/s 2024-09-12T03:08:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:35 smithi181 ceph-mon[35320]: pgmap v82: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 200 MiB/s wr, 1.87k op/s 2024-09-12T03:08:37.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:37 smithi057 ceph-mon[30361]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 162 MiB/s wr, 1.49k op/s 2024-09-12T03:08:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:37 smithi204 ceph-mon[35013]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 162 MiB/s wr, 1.49k op/s 2024-09-12T03:08:37.702 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:37 smithi181 ceph-mon[35320]: pgmap v83: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 162 MiB/s wr, 1.49k op/s 2024-09-12T03:08:39.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:39 smithi057 ceph-mon[30361]: pgmap v84: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 2.20k op/s 2024-09-12T03:08:39.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:39 smithi204 ceph-mon[35013]: pgmap v84: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 2.20k op/s 2024-09-12T03:08:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:39 smithi181 ceph-mon[35320]: pgmap v84: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 259 MiB/s wr, 2.20k op/s 2024-09-12T03:08:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.604+0000 7ff6e0400640 1 -- 172.21.15.57:0/1025473819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 msgr2=0x7ff6d8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.604+0000 7ff6e0400640 1 --2- 172.21.15.57:0/1025473819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d8110250 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009930 tx=0x7ff6cc02a480 comp rx=0 tx=0).stop 2024-09-12T03:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.605+0000 7ff6e0400640 1 -- 172.21.15.57:0/1025473819 shutdown_connections 2024-09-12T03:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.605+0000 7ff6e0400640 1 --2- 172.21.15.57:0/1025473819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d811b140 0x7ff6d811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.605+0000 7ff6e0400640 1 --2- 172.21.15.57:0/1025473819 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d81095d0 0x7ff6d8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.605+0000 7ff6e0400640 1 --2- 172.21.15.57:0/1025473819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d8110250 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.606+0000 7ff6e0400640 1 -- 172.21.15.57:0/1025473819 >> 172.21.15.57:0/1025473819 conn(0x7ff6d806ce00 msgr2=0x7ff6d806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.606+0000 7ff6e0400640 1 -- 172.21.15.57:0/1025473819 shutdown_connections 2024-09-12T03:08:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.606+0000 7ff6e0400640 1 -- 172.21.15.57:0/1025473819 wait complete. 2024-09-12T03:08:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.608+0000 7ff6e0400640 1 Processor -- start 2024-09-12T03:08:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.609+0000 7ff6e0400640 1 -- start start 2024-09-12T03:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.610+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d81a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.610+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.610+0000 7ff6e0400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 0x7ff6d81ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6e0400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8120cf0 con 0x7ff6d811b140 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6e0400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8120b70 con 0x7ff6d81095d0 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6dd000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38450/0 (socket says 172.21.15.57:38450) 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6dd000640 1 -- 172.21.15.57:0/3452333988 learned_addr learned my addr 172.21.15.57:0/3452333988 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6e0400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8120e70 con 0x7ff6d81091f0 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.611+0000 7ff6de400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 0x7ff6d81ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.612+0000 7ff6dda00640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d81a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6dd000640 1 -- 172.21.15.57:0/3452333988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 msgr2=0x7ff6d81a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6dd000640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d81a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6dd000640 1 -- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 msgr2=0x7ff6d81ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6dd000640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 0x7ff6d81ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6dd000640 1 -- 172.21.15.57:0/3452333988 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d81abe90 con 0x7ff6d81095d0 2024-09-12T03:08:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.613+0000 7ff6de400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 0x7ff6d81ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.614+0000 7ff6dd000640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400c920 tx=0x7ff6c400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.615+0000 7ff6d6000640 1 -- 172.21.15.57:0/3452333988 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4007c70 con 0x7ff6d81095d0 2024-09-12T03:08:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.616+0000 7ff6d6000640 1 -- 172.21.15.57:0/3452333988 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6c4007e10 con 0x7ff6d81095d0 2024-09-12T03:08:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.616+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d81ac180 con 0x7ff6d81095d0 2024-09-12T03:08:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.616+0000 7ff6d6000640 1 -- 172.21.15.57:0/3452333988 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4018890 con 0x7ff6d81095d0 2024-09-12T03:08:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.616+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d81b3a10 con 0x7ff6d81095d0 2024-09-12T03:08:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.627+0000 7ff6e32a6740 1 Processor -- start 2024-09-12T03:08:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.627+0000 7ff6d6000640 1 -- 172.21.15.57:0/3452333988 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ff6c4018a50 con 0x7ff6d81095d0 2024-09-12T03:08:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.628+0000 7ff6e32a6740 1 -- start start 2024-09-12T03:08:40.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.628+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6d1e0 0x55c65ce6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c65cdd3100 con 0x55c65ce6d1e0 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c65cd7b1b0 con 0x55c65ce783a0 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c65cdcf0c0 con 0x55c65ce6db00 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.629+0000 7ff6de400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6d1e0 0x55c65ce6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.630+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.630+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:57232/0 (socket says 172.21.15.57:57232) 2024-09-12T03:08:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.630+0000 7ff6dda00640 1 -- 172.21.15.57:0/2852921708 learned_addr learned my addr 172.21.15.57:0/2852921708 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.630+0000 7ff6d6000640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 0x7ff6c00a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dd000640 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 -- 172.21.15.57:0/2852921708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 msgr2=0x55c65ce7a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce7a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 -- 172.21.15.57:0/2852921708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6d1e0 msgr2=0x55c65ce6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6d1e0 0x55c65ce6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 -- 172.21.15.57:0/2852921708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65cae6210 con 0x55c65ce6db00 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6d6000640 1 -- 172.21.15.57:0/3452333988 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff6c40cf690 con 0x7ff6d81095d0 2024-09-12T03:08:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.631+0000 7ff6dda00640 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc033000 tx=0x7ff6cc00b160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8778aec96a55f940 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.632+0000 7ff6de400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 0x7ff6c00a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.632+0000 7ff6caa00640 1 -- 172.21.15.57:0/2852921708 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc035070 con 0x55c65ce6db00 2024-09-12T03:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.632+0000 7ff6caa00640 1 -- 172.21.15.57:0/2852921708 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc032bb0 con 0x55c65ce6db00 2024-09-12T03:08:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.633+0000 7ff6e32a6740 1 -- 172.21.15.57:0/2852921708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 msgr2=0x55c65ce77e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.634+0000 7ff6de400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 0x7ff6c00a5490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007fd0 tx=0x7ff6d0004a40 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.635+0000 7ff6caa00640 1 -- 172.21.15.57:0/2852921708 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc035070 con 0x55c65ce6db00 2024-09-12T03:08:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc033000 tx=0x7ff6cc00b160 comp rx=0 tx=0).stop 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 -- 172.21.15.57:0/2852921708 shutdown_connections 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce7a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce77e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/2852921708 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6d1e0 0x55c65ce6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 -- 172.21.15.57:0/2852921708 >> 172.21.15.57:0/2852921708 conn(0x55c65ce69750 msgr2=0x55c65ce69b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 -- 172.21.15.57:0/2852921708 shutdown_connections 2024-09-12T03:08:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.636+0000 7ff6e32a6740 1 -- 172.21.15.57:0/2852921708 wait complete. 2024-09-12T03:08:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.638+0000 7ff6e32a6740 1 -- start start 2024-09-12T03:08:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.638+0000 7ff6e32a6740 1 Processor -- start 2024-09-12T03:08:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.641+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.645+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.645+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c65ce7afc0 con 0x55c65ce783a0 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c65cd5c090 con 0x55c65ce6d1e0 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c65ce7b140 con 0x55c65ce6db00 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6e32a6740 10 client.0 _ll_register_callbacks cb 0x55c65cdb0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6dda00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38468/0 (socket says 172.21.15.57:38468) 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.646+0000 7ff6dda00640 1 -- 172.21.15.57:0/3896590370 learned_addr learned my addr 172.21.15.57:0/3896590370 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.647+0000 7ff6de400640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.648+0000 7ff6dda00640 1 -- 172.21.15.57:0/3896590370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 msgr2=0x55c65ce75750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.648+0000 7ff6dda00640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce75750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.648+0000 7ff6dda00640 1 -- 172.21.15.57:0/3896590370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 msgr2=0x55c65ce75c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.648+0000 7ff6dda00640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce75c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.648+0000 7ff6dda00640 1 -- 172.21.15.57:0/3896590370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65cbb2620 con 0x55c65ce6d1e0 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.650+0000 7ff6dda00640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009850 tx=0x7ff6cc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9ac99903e853b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.650+0000 7ff6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:08:40.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.652+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc035040 con 0x55c65ce6d1e0 2024-09-12T03:08:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.652+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff6cc003e00 con 0x55c65ce6d1e0 2024-09-12T03:08:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.654+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6cc032bb0 con 0x55c65ce6d1e0 2024-09-12T03:08:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.656+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6cc0036a0 con 0x55c65ce6d1e0 2024-09-12T03:08:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.657+0000 7ff6e32a6740 10 client.25454 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.657+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c65ce86b30 con 0x55c65ce6d1e0 2024-09-12T03:08:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.657+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6cc032bb0 con 0x55c65ce6d1e0 2024-09-12T03:08:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.659+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6cc043660 con 0x55c65ce6d1e0 2024-09-12T03:08:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.659+0000 7ff6e32a6740 10 client.25454 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.659+0000 7ff6e32a6740 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.659+0000 7ff6e32a6740 20 client.25454 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.659+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65ce87e80 0x55c65ce8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.660+0000 7ff6e32a6740 4 client.25454 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.660+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c65cde0830 con 0x55c65ce87e80 2024-09-12T03:08:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.660+0000 7ff6dd000640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65ce87e80 0x55c65ce8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.661+0000 7ff6dd000640 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65ce87e80 0x55c65ce8a270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2da659ab5231894 server_cookie=e802dd99883d5afe in_seq=0 out_seq=0 2024-09-12T03:08:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.662+0000 7ff6bf400640 10 client.25454 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.662+0000 7ff6bf400640 1 -- 172.21.15.57:0/3896590370 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c65cde0830 con 0x55c65ce87e80 2024-09-12T03:08:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.662+0000 7ff6bf400640 10 client.25454 handle_command_reply: tid=0 2024-09-12T03:08:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.663+0000 7ff6e32a6740 1 client.25454 shutdown 2024-09-12T03:08:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.664+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 msgr2=0x55c65ce75210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.664+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009850 tx=0x7ff6cc005850 comp rx=0 tx=0).stop 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 shutdown_connections 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65ce87e80 0x55c65ce8a270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce6db00 0x55c65ce75750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3896590370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6d1e0 0x55c65ce75210 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009850 tx=0x7ff6cc005850 comp rx=0 tx=0).stop 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6bf400640 0 client.25454 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.665+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 >> 172.21.15.57:0/3896590370 conn(0x55c65ce69750 msgr2=0x55c65ce6f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.666+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 shutdown_connections 2024-09-12T03:08:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.666+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3896590370 wait complete. 2024-09-12T03:08:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.666+0000 7ff6e32a6740 20 client.25454 trim_cache size 0 max 16384 2024-09-12T03:08:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 Processor -- start 2024-09-12T03:08:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 -- start start 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6db00 0x55c65ce68720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8012870 con 0x55c65ce6db00 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b8010140 con 0x55c65ce783a0 2024-09-12T03:08:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.700+0000 7ff6e32a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b800df90 con 0x55c65ce68c60 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:57250/0 (socket says 172.21.15.57:57250) 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 -- 172.21.15.57:0/3780116139 learned_addr learned my addr 172.21.15.57:0/3780116139 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 -- 172.21.15.57:0/3780116139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 msgr2=0x55c65ce8f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 -- 172.21.15.57:0/3780116139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6db00 msgr2=0x55c65ce68720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6db00 0x55c65ce68720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.701+0000 7ff6de400640 1 -- 172.21.15.57:0/3780116139 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65ce84220 con 0x55c65ce68c60 2024-09-12T03:08:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.702+0000 7ff6de400640 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000d9c0 tx=0x7ff6d0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb6b04189044cca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.705+0000 7ff6ca000640 1 -- 172.21.15.57:0/3780116139 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0012b60 con 0x55c65ce68c60 2024-09-12T03:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.705+0000 7ff6ca000640 1 -- 172.21.15.57:0/3780116139 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d0016070 con 0x55c65ce68c60 2024-09-12T03:08:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.705+0000 7ff6ca000640 1 -- 172.21.15.57:0/3780116139 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002c6a0 con 0x55c65ce68c60 2024-09-12T03:08:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3780116139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 msgr2=0x55c65ce8fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000d9c0 tx=0x7ff6d0014000 comp rx=0 tx=0).stop 2024-09-12T03:08:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3780116139 shutdown_connections 2024-09-12T03:08:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ce68c60 0x55c65ce8fd80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce783a0 0x55c65ce8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/3780116139 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce6db00 0x55c65ce68720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3780116139 >> 172.21.15.57:0/3780116139 conn(0x55c65ce69750 msgr2=0x55c65ce68010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3780116139 shutdown_connections 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.706+0000 7ff6e32a6740 1 -- 172.21.15.57:0/3780116139 wait complete. 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.707+0000 7ff6e32a6740 1 -- start start 2024-09-12T03:08:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.707+0000 7ff6e32a6740 1 Processor -- start 2024-09-12T03:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6db00 0x55c65ce7c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce7ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6e32a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6de400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6de400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:57252/0 (socket says 172.21.15.57:57252) 2024-09-12T03:08:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.708+0000 7ff6de400640 1 -- 172.21.15.57:0/212380620 learned_addr learned my addr 172.21.15.57:0/212380620 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b800e600 con 0x55c65ce783a0 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b800fa70 con 0x55c65ce6db00 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b800ecd0 con 0x55c65ceeb7e0 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6dda00640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6db00 0x55c65ce7c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6e32a6740 10 client.0 _ll_register_callbacks cb 0x7ff6cc00b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6dd000640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce7ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6de400640 1 -- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6db00 msgr2=0x55c65ce7c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6de400640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6db00 0x55c65ce7c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6de400640 1 -- 172.21.15.57:0/212380620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 msgr2=0x55c65ce7ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6de400640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce7ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.709+0000 7ff6de400640 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65ceec1d0 con 0x55c65ceeb7e0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.710+0000 7ff6de400640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007560 tx=0x7ff6d002e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3865cbcce4b78d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.710+0000 7ff6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.710+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000d640 con 0x55c65ceeb7e0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.710+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff6d0038410 con 0x55c65ceeb7e0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.710+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d0016070 con 0x55c65ceeb7e0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.711+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6d0038610 con 0x55c65ceeb7e0 2024-09-12T03:08:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.711+0000 7ff6e32a6740 10 client.25480 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.711+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c65cef3520 con 0x55c65ceeb7e0 2024-09-12T03:08:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.711+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d0016070 con 0x55c65ceeb7e0 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6d004b990 con 0x55c65ceeb7e0 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 10 client.25480 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 10 client.25480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 20 client.25480 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65cef3b70 0x55c65cef5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 4 client.25480 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:08:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.712+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c65cddab80 con 0x55c65cef3b70 2024-09-12T03:08:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.713+0000 7ff6dda00640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65cef3b70 0x55c65cef5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.714+0000 7ff6bf400640 10 client.25480 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.714+0000 7ff6dda00640 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65cef3b70 0x55c65cef5f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce039eda67591e78 server_cookie=934cc360dc2d7fad in_seq=0 out_seq=0 2024-09-12T03:08:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.723+0000 7ff6bf400640 1 -- 172.21.15.57:0/212380620 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c65cddab80 con 0x55c65cef3b70 2024-09-12T03:08:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.723+0000 7ff6bf400640 10 client.25480 handle_command_reply: tid=0 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.723+0000 7ff6e32a6740 1 client.25480 shutdown 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.723+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 msgr2=0x55c65ceebc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.723+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007560 tx=0x7ff6d002e730 comp rx=0 tx=0).stop 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 shutdown_connections 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c65cef3b70 0x55c65cef5f90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c65ceeb7e0 0x55c65ceebc60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c65ce783a0 0x55c65ce7ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 --2- 172.21.15.57:0/212380620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c65ce6db00 0x55c65ce7c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 >> 172.21.15.57:0/212380620 conn(0x55c65ce69750 msgr2=0x55c65ce6ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6bf400640 0 client.25480 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 shutdown_connections 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 1 -- 172.21.15.57:0/212380620 wait complete. 2024-09-12T03:08:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.724+0000 7ff6e32a6740 20 client.25480 trim_cache size 0 max 16384 2024-09-12T03:08:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:08:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:08:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:08:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 msgr2=0x7ff6c00a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 0x7ff6c00a5490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007fd0 tx=0x7ff6d0004a40 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 msgr2=0x7ff6d81a73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400c920 tx=0x7ff6c400cde0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 shutdown_connections 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff6c00a2fd0 0x7ff6c00a5490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff6d811b140 0x7ff6d81ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d81095d0 0x7ff6d81a73e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 --2- 172.21.15.57:0/3452333988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d81091f0 0x7ff6d81a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.725+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 >> 172.21.15.57:0/3452333988 conn(0x7ff6d806ce00 msgr2=0x7ff6d8072760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.726+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 shutdown_connections 2024-09-12T03:08:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:40.726+0000 7ff6e0400640 1 -- 172.21.15.57:0/3452333988 wait complete. 2024-09-12T03:08:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:41 smithi057 ceph-mon[30361]: pgmap v85: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 186 MiB/s wr, 1.53k op/s 2024-09-12T03:08:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:41 smithi204 ceph-mon[35013]: pgmap v85: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 186 MiB/s wr, 1.53k op/s 2024-09-12T03:08:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:41 smithi181 ceph-mon[35320]: pgmap v85: 129 pgs: 129 active+clean; 12 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 186 MiB/s wr, 1.53k op/s 2024-09-12T03:08:43.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:43 smithi057 ceph-mon[30361]: pgmap v86: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.90k op/s 2024-09-12T03:08:43.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:43 smithi204 ceph-mon[35013]: pgmap v86: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.90k op/s 2024-09-12T03:08:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:43 smithi181 ceph-mon[35320]: pgmap v86: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 248 MiB/s wr, 1.90k op/s 2024-09-12T03:08:45.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:45 smithi057 ceph-mon[30361]: pgmap v87: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 225 MiB/s wr, 1.57k op/s 2024-09-12T03:08:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:45 smithi204 ceph-mon[35013]: pgmap v87: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 225 MiB/s wr, 1.57k op/s 2024-09-12T03:08:45.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:45 smithi181 ceph-mon[35320]: pgmap v87: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 225 MiB/s wr, 1.57k op/s 2024-09-12T03:08:47.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:47 smithi057 ceph-mon[30361]: pgmap v88: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-12T03:08:47.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:47 smithi204 ceph-mon[35013]: pgmap v88: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-12T03:08:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:47 smithi181 ceph-mon[35320]: pgmap v88: 129 pgs: 129 active+clean; 14 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-12T03:08:49.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:49 smithi057 ceph-mon[30361]: pgmap v89: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 295 MiB/s wr, 1.79k op/s 2024-09-12T03:08:49.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:49 smithi204 ceph-mon[35013]: pgmap v89: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 295 MiB/s wr, 1.79k op/s 2024-09-12T03:08:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:49 smithi181 ceph-mon[35320]: pgmap v89: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 295 MiB/s wr, 1.79k op/s 2024-09-12T03:08:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:51 smithi204 ceph-mon[35013]: pgmap v90: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 198 MiB/s wr, 1.08k op/s 2024-09-12T03:08:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:51 smithi181 ceph-mon[35320]: pgmap v90: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 198 MiB/s wr, 1.08k op/s 2024-09-12T03:08:51.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:51 smithi057 ceph-mon[30361]: pgmap v90: 129 pgs: 129 active+clean; 15 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 198 MiB/s wr, 1.08k op/s 2024-09-12T03:08:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:53 smithi204 ceph-mon[35013]: pgmap v91: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 1.38k op/s 2024-09-12T03:08:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:53 smithi181 ceph-mon[35320]: pgmap v91: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 1.38k op/s 2024-09-12T03:08:53.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:53 smithi057 ceph-mon[30361]: pgmap v91: 129 pgs: 129 active+clean; 15 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 268 MiB/s wr, 1.38k op/s 2024-09-12T03:08:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.979+0000 7fd1fec00640 1 -- 172.21.15.57:0/309029853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f8112920 msgr2=0x7fd1f8114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.979+0000 7fd1fec00640 1 --2- 172.21.15.57:0/309029853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f8112920 0x7fd1f8114dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009ca0 tx=0x7fd1f002a8d0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.980+0000 7fd1fec00640 1 -- 172.21.15.57:0/309029853 shutdown_connections 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.980+0000 7fd1fec00640 1 --2- 172.21.15.57:0/309029853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f8112920 0x7fd1f8114dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.980+0000 7fd1fec00640 1 --2- 172.21.15.57:0/309029853 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f8109e10 0x7fd1f8112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.980+0000 7fd1fec00640 1 --2- 172.21.15.57:0/309029853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f8109460 0x7fd1f8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.980+0000 7fd1fec00640 1 -- 172.21.15.57:0/309029853 >> 172.21.15.57:0/309029853 conn(0x7fd1f806d040 msgr2=0x7fd1f806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.981+0000 7fd1fec00640 1 -- 172.21.15.57:0/309029853 shutdown_connections 2024-09-12T03:08:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.981+0000 7fd1fec00640 1 -- 172.21.15.57:0/309029853 wait complete. 2024-09-12T03:08:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.983+0000 7fd1fec00640 1 Processor -- start 2024-09-12T03:08:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.983+0000 7fd1fec00640 1 -- start start 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.985+0000 7fd1fec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f8112920 0x7fd1f81a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.985+0000 7fd1fec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.985+0000 7fd1fec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f819f250 0x7fd1f81a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.986+0000 7fd1fec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81183b0 con 0x7fd1f8112920 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.986+0000 7fd1fec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8118230 con 0x7fd1f819e8b0 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.986+0000 7fd1fec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8118530 con 0x7fd1f819f250 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.987+0000 7fd1fce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.987+0000 7fd1fce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:33912/0 (socket says 172.21.15.57:33912) 2024-09-12T03:08:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.987+0000 7fd1fce00640 1 -- 172.21.15.57:0/236903585 learned_addr learned my addr 172.21.15.57:0/236903585 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.987+0000 7fd1fce00640 1 -- 172.21.15.57:0/236903585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f819f250 msgr2=0x7fd1f81a3350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.987+0000 7fd1fce00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f819f250 0x7fd1f81a3350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.988+0000 7fd1fce00640 1 -- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f8112920 msgr2=0x7fd1f81a5830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.988+0000 7fd1fce00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f8112920 0x7fd1f81a5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.988+0000 7fd1fce00640 1 -- 172.21.15.57:0/236903585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f81a3890 con 0x7fd1f819e8b0 2024-09-12T03:08:53.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.989+0000 7fd1fce00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b4a0 tx=0x7fd1ec00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:53.992 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.992+0000 7fd1f6000640 1 -- 172.21.15.57:0/236903585 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec011830 con 0x7fd1f819e8b0 2024-09-12T03:08:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.992+0000 7fd1f6000640 1 -- 172.21.15.57:0/236903585 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec016070 con 0x7fd1f819e8b0 2024-09-12T03:08:53.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.992+0000 7fd1f6000640 1 -- 172.21.15.57:0/236903585 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00f2e0 con 0x7fd1f819e8b0 2024-09-12T03:08:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.996+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f81a3b20 con 0x7fd1f819e8b0 2024-09-12T03:08:53.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:53.996+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a4020 con 0x7fd1f819e8b0 2024-09-12T03:08:54.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.006+0000 7fd20307f740 1 Processor -- start 2024-09-12T03:08:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.006+0000 7fd1f6000640 1 -- 172.21.15.57:0/236903585 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fd1ec01a410 con 0x7fd1f819e8b0 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.006+0000 7fd20307f740 1 -- start start 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 0x55be92a15030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a17960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55be92970100 con 0x55be92a0acd0 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55be929181b0 con 0x55be92a0a3b0 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.007+0000 7fd20307f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be9296c0c0 con 0x55be92a15570 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.008+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.008+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:33922/0 (socket says 172.21.15.57:33922) 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.008+0000 7fd1fe200640 1 -- 172.21.15.57:0/3863560 learned_addr learned my addr 172.21.15.57:0/3863560 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.008+0000 7fd1fd800640 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 0x55be92a15030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.009+0000 7fd1fe200640 1 -- 172.21.15.57:0/3863560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 msgr2=0x55be92a17960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.009+0000 7fd1fe200640 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a17960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.009+0000 7fd1fe200640 1 -- 172.21.15.57:0/3863560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 msgr2=0x55be92a15030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.009+0000 7fd1fe200640 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 0x55be92a15030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.009+0000 7fd1fe200640 1 -- 172.21.15.57:0/3863560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be92683210 con 0x55be92a0a3b0 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.010+0000 7fd1f6000640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 0x7fd1e00a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.010+0000 7fd1f6000640 1 -- 172.21.15.57:0/236903585 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd1ec0ce4d0 con 0x7fd1f819e8b0 2024-09-12T03:08:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.010+0000 7fd1fd800640 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 0x55be92a15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:08:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.012+0000 7fd1fd800640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 0x7fd1e00a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.012+0000 7fd1fe200640 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd1f002f500 tx=0x7fd1f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7240bd8ea7b87421 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.012+0000 7fd1eaa00640 1 -- 172.21.15.57:0/3863560 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0031d80 con 0x55be92a0a3b0 2024-09-12T03:08:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.012+0000 7fd1eaa00640 1 -- 172.21.15.57:0/3863560 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0061070 con 0x55be92a0a3b0 2024-09-12T03:08:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.012+0000 7fd1eaa00640 1 -- 172.21.15.57:0/3863560 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00042a0 con 0x55be92a0a3b0 2024-09-12T03:08:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.013+0000 7fd20307f740 1 -- 172.21.15.57:0/3863560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 msgr2=0x55be92a0a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.013+0000 7fd20307f740 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd1f002f500 tx=0x7fd1f0009780 comp rx=0 tx=0).stop 2024-09-12T03:08:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.015+0000 7fd20307f740 1 -- 172.21.15.57:0/3863560 shutdown_connections 2024-09-12T03:08:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.015+0000 7fd20307f740 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a17960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.015+0000 7fd20307f740 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0acd0 0x55be92a15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.016+0000 7fd20307f740 1 --2- 172.21.15.57:0/3863560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0a3b0 0x55be92a0a790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.016+0000 7fd20307f740 1 -- 172.21.15.57:0/3863560 >> 172.21.15.57:0/3863560 conn(0x55be92a06920 msgr2=0x55be92a06d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.016+0000 7fd1fd800640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 0x7fd1e00a5360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55be92a07780 tx=0x7fd1e4009610 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.016+0000 7fd20307f740 1 -- 172.21.15.57:0/3863560 shutdown_connections 2024-09-12T03:08:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.016+0000 7fd20307f740 1 -- 172.21.15.57:0/3863560 wait complete. 2024-09-12T03:08:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.017+0000 7fd20307f740 1 -- start start 2024-09-12T03:08:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.018+0000 7fd20307f740 1 Processor -- start 2024-09-12T03:08:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.019+0000 7fd20307f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.019+0000 7fd20307f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0acd0 0x55be92a12b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.019+0000 7fd20307f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.020+0000 7fd20307f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55be92a18050 con 0x55be92a0a3b0 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.020+0000 7fd20307f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55be928f9090 con 0x55be92a0acd0 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.020+0000 7fd20307f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be92a181d0 con 0x55be92a15570 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.020+0000 7fd20307f740 10 client.0 _ll_register_callbacks cb 0x55be9294df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.021+0000 7fd1fd800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.021+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.021+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60894/0 (socket says 172.21.15.57:60894) 2024-09-12T03:08:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.021+0000 7fd1fe200640 1 -- 172.21.15.57:0/4219779405 learned_addr learned my addr 172.21.15.57:0/4219779405 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.022+0000 7fd1fe200640 1 -- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0acd0 msgr2=0x55be92a12b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.022+0000 7fd1fce00640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0acd0 0x55be92a12b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.022+0000 7fd1fe200640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0acd0 0x55be92a12b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.022+0000 7fd1fe200640 1 -- 172.21.15.57:0/4219779405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 msgr2=0x55be92a125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.023+0000 7fd1fe200640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.023+0000 7fd1fe200640 1 -- 172.21.15.57:0/4219779405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be9274f620 con 0x55be92a15570 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.023+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f0061070 con 0x55be92a15570 2024-09-12T03:08:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.025+0000 7fd1fe200640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097f0 tx=0x7fd1f0003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6bdb0f34475245 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.025+0000 7fd1df400640 10 client.25486 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:08:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.032+0000 7fd1df400640 1 -- 172.21.15.57:0/4219779405 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004240 con 0x55be92a15570 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.038+0000 7fd1df400640 1 -- 172.21.15.57:0/4219779405 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd1f00043e0 con 0x55be92a15570 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.039+0000 7fd1df400640 1 -- 172.21.15.57:0/4219779405 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1f00045e0 con 0x55be92a15570 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.039+0000 7fd20307f740 10 client.25486 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.039+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55be92a18050 con 0x55be92a15570 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd1df400640 1 -- 172.21.15.57:0/4219779405 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1f002f620 con 0x55be92a15570 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 10 client.25486 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 20 client.25486 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a24fb0 0x55be92a273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 4 client.25486 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:08:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.040+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be9297d830 con 0x55be92a24fb0 2024-09-12T03:08:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.041+0000 7fd1fd800640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a24fb0 0x55be92a273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.041+0000 7fd1fd800640 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a24fb0 0x55be92a273a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c5de9f4c20c040 server_cookie=3b7d976376c66d95 in_seq=0 out_seq=0 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd1df400640 10 client.25486 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd1df400640 1 -- 172.21.15.57:0/4219779405 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be9297d830 con 0x55be92a24fb0 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd1df400640 10 client.25486 handle_command_reply: tid=0 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd20307f740 1 client.25486 shutdown 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 msgr2=0x55be92a13070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.046+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00097f0 tx=0x7fd1f0003ba0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 shutdown_connections 2024-09-12T03:08:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a24fb0 0x55be92a273a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a15570 0x55be92a13070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a0acd0 0x55be92a12b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 --2- 172.21.15.57:0/4219779405 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd1df400640 0 client.25486 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 >> 172.21.15.57:0/4219779405 conn(0x55be92a06920 msgr2=0x55be92a0c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 shutdown_connections 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.047+0000 7fd20307f740 1 -- 172.21.15.57:0/4219779405 wait complete. 2024-09-12T03:08:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.048+0000 7fd20307f740 20 client.25486 trim_cache size 0 max 16384 2024-09-12T03:08:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.093+0000 7fd20307f740 1 Processor -- start 2024-09-12T03:08:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.093+0000 7fd20307f740 1 -- start start 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a2c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a0be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81b2380 con 0x55be92a0a3b0 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81afc50 con 0x55be92a15570 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.096+0000 7fd20307f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8073c80 con 0x55be92a0acd0 2024-09-12T03:08:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.097+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a0be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.097+0000 7fd1fe200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a0be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:33930/0 (socket says 172.21.15.57:33930) 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.097+0000 7fd1fce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.097+0000 7fd1fce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60902/0 (socket says 172.21.15.57:60902) 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.097+0000 7fd1fce00640 1 -- 172.21.15.57:0/3522248222 learned_addr learned my addr 172.21.15.57:0/3522248222 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.098+0000 7fd1fce00640 1 -- 172.21.15.57:0/3522248222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 msgr2=0x55be92a0be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.098+0000 7fd1fce00640 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a0be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.098+0000 7fd1fce00640 1 -- 172.21.15.57:0/3522248222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 msgr2=0x55be92a2c9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:08:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.099+0000 7fd1fce00640 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a2c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.099+0000 7fd1fce00640 1 -- 172.21.15.57:0/3522248222 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be92a21420 con 0x55be92a0acd0 2024-09-12T03:08:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.099+0000 7fd1fce00640 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0971b0 tx=0x7fd1ec09a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e3a53a3f5ab76d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.100+0000 7fd1ea000640 1 -- 172.21.15.57:0/3522248222 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0a10c0 con 0x55be92a0acd0 2024-09-12T03:08:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.100+0000 7fd1ea000640 1 -- 172.21.15.57:0/3522248222 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec09aee0 con 0x55be92a0acd0 2024-09-12T03:08:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.100+0000 7fd1ea000640 1 -- 172.21.15.57:0/3522248222 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0a10c0 con 0x55be92a0acd0 2024-09-12T03:08:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.102+0000 7fd20307f740 1 -- 172.21.15.57:0/3522248222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 msgr2=0x55be92a2cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.102+0000 7fd20307f740 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0971b0 tx=0x7fd1ec09a140 comp rx=0 tx=0).stop 2024-09-12T03:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 -- 172.21.15.57:0/3522248222 shutdown_connections 2024-09-12T03:08:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a0be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a2cee0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 --2- 172.21.15.57:0/3522248222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 -- 172.21.15.57:0/3522248222 >> 172.21.15.57:0/3522248222 conn(0x55be92a06920 msgr2=0x55be92a0d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 -- 172.21.15.57:0/3522248222 shutdown_connections 2024-09-12T03:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.103+0000 7fd20307f740 1 -- 172.21.15.57:0/3522248222 wait complete. 2024-09-12T03:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.104+0000 7fd20307f740 1 -- start start 2024-09-12T03:08:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.104+0000 7fd20307f740 1 Processor -- start 2024-09-12T03:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.106+0000 7fd20307f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81ae110 con 0x55be92a0a3b0 2024-09-12T03:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81af580 con 0x55be92a15570 2024-09-12T03:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81ae7e0 con 0x55be92a0acd0 2024-09-12T03:08:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd20307f740 10 client.0 _ll_register_callbacks cb 0x7fd1e4006e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.107+0000 7fd1fd800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40818/0 (socket says 172.21.15.57:40818) 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 -- 172.21.15.57:0/2890417798 learned_addr learned my addr 172.21.15.57:0/2890417798 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fe200640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fce00640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 -- 172.21.15.57:0/2890417798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 msgr2=0x55be92a0fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a0fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 -- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 msgr2=0x55be92a100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.108+0000 7fd1fd800640 1 -- 172.21.15.57:0/2890417798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be92a8e9c0 con 0x55be92a0a3b0 2024-09-12T03:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.109+0000 7fd1fe200640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:08:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.109+0000 7fd1fd800640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8073a10 tx=0x7fd1e4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf2f611f25ae61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:08:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.111+0000 7fd1df400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:08:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.112+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4015690 con 0x55be92a0a3b0 2024-09-12T03:08:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.113+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd1e400f040 con 0x55be92a0a3b0 2024-09-12T03:08:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.113+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1ec09aee0 con 0x55be92a0a3b0 2024-09-12T03:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.121+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1e4037050 con 0x55be92a0a3b0 2024-09-12T03:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.121+0000 7fd20307f740 10 client.15399 fetch_fsmap learned FSMap version 28 2024-09-12T03:08:54.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.121+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be92a8fcb0 con 0x55be92a0a3b0 2024-09-12T03:08:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.121+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e402e070 con 0x55be92a0a3b0 2024-09-12T03:08:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.123+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1e40139d0 con 0x55be92a0a3b0 2024-09-12T03:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.123+0000 7fd20307f740 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.123+0000 7fd20307f740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.124+0000 7fd20307f740 20 client.15399 populate_metadata read hostname 'smithi057' 2024-09-12T03:08:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.124+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a903d0 0x55be92a927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:08:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.124+0000 7fd20307f740 4 client.15399 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:08:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.124+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd1ec09aee0 con 0x55be92a903d0 2024-09-12T03:08:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.124+0000 7fd1fce00640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a903d0 0x55be92a927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:08:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.127+0000 7fd1fce00640 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a903d0 0x55be92a927c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d9c92ab6e63615 server_cookie=64f7e99d0c368e32 in_seq=0 out_seq=0 2024-09-12T03:08:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.127+0000 7fd1df400640 10 client.15399 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.206+0000 7fd1df400640 1 -- 172.21.15.57:0/2890417798 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fd1ec09aee0 con 0x55be92a903d0 2024-09-12T03:08:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.206+0000 7fd1df400640 10 client.15399 handle_command_reply: tid=0 2024-09-12T03:08:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.206+0000 7fd20307f740 1 client.15399 shutdown 2024-09-12T03:08:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.217+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 msgr2=0x55be92a0f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.217+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8073a10 tx=0x7fd1e4009890 comp rx=0 tx=0).stop 2024-09-12T03:08:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.219+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 shutdown_connections 2024-09-12T03:08:54.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.219+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55be92a903d0 0x55be92a927c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55be92a15570 0x55be92a100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be92a0acd0 0x55be92a0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd20307f740 1 --2- 172.21.15.57:0/2890417798 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55be92a0a3b0 0x55be92a0f630 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 >> 172.21.15.57:0/2890417798 conn(0x55be92a06920 msgr2=0x55be92a125b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd1df400640 0 client.15399 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.220+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 shutdown_connections 2024-09-12T03:08:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.221+0000 7fd20307f740 1 -- 172.21.15.57:0/2890417798 wait complete. 2024-09-12T03:08:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.222+0000 7fd20307f740 20 client.15399 trim_cache size 0 max 16384 2024-09-12T03:08:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:08:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:08:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:08:54.225 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:08:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.226+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 msgr2=0x7fd1e00a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.226+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 0x7fd1e00a5360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x55be92a07780 tx=0x7fd1e4009610 comp rx=0 tx=0).stop 2024-09-12T03:08:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.226+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 msgr2=0x7fd1f819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:08:54.227 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.226+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b4a0 tx=0x7fd1ec00b960 comp rx=0 tx=0).stop 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 shutdown_connections 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f819f250 0x7fd1f81a3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1f819e8b0 0x7fd1f819ed10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd1e00a2ea0 0x7fd1e00a5360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 --2- 172.21.15.57:0/236903585 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd1f8112920 0x7fd1f81a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.227+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 >> 172.21.15.57:0/236903585 conn(0x7fd1f806d040 msgr2=0x7fd1f8112d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:08:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.228+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 shutdown_connections 2024-09-12T03:08:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:08:54.229+0000 7fd1fec00640 1 -- 172.21.15.57:0/236903585 wait complete. 2024-09-12T03:08:55.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:55 smithi204 ceph-mon[35013]: pgmap v92: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 241 MiB/s wr, 1.16k op/s 2024-09-12T03:08:55.673 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:55 smithi057 ceph-mon[30361]: pgmap v92: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 241 MiB/s wr, 1.16k op/s 2024-09-12T03:08:55.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:55 smithi181 ceph-mon[35320]: pgmap v92: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 241 MiB/s wr, 1.16k op/s 2024-09-12T03:08:57.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:57 smithi181 ceph-mon[35320]: pgmap v93: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 205 MiB/s wr, 948 op/s 2024-09-12T03:08:57.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:57 smithi057 ceph-mon[30361]: pgmap v93: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 205 MiB/s wr, 948 op/s 2024-09-12T03:08:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:57 smithi204 ceph-mon[35013]: pgmap v93: 129 pgs: 129 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 205 MiB/s wr, 948 op/s 2024-09-12T03:08:59.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:08:59 smithi181 ceph-mon[35320]: pgmap v94: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 312 MiB/s wr, 1.40k op/s 2024-09-12T03:08:59.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:08:59 smithi057 ceph-mon[30361]: pgmap v94: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 312 MiB/s wr, 1.40k op/s 2024-09-12T03:08:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:08:59 smithi204 ceph-mon[35013]: pgmap v94: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 312 MiB/s wr, 1.40k op/s 2024-09-12T03:09:01.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:01 smithi181 ceph-mon[35320]: pgmap v95: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 211 MiB/s wr, 907 op/s 2024-09-12T03:09:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:01 smithi057 ceph-mon[30361]: pgmap v95: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 211 MiB/s wr, 907 op/s 2024-09-12T03:09:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:01 smithi204 ceph-mon[35013]: pgmap v95: 129 pgs: 129 active+clean; 17 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 211 MiB/s wr, 907 op/s 2024-09-12T03:09:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:03 smithi057 ceph-mon[30361]: pgmap v96: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 282 MiB/s wr, 1.24k op/s 2024-09-12T03:09:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:03 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:03 smithi204 ceph-mon[35013]: pgmap v96: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 282 MiB/s wr, 1.24k op/s 2024-09-12T03:09:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:03 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:03 smithi181 ceph-mon[35320]: pgmap v96: 129 pgs: 129 active+clean; 17 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 282 MiB/s wr, 1.24k op/s 2024-09-12T03:09:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:03 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:05.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:05 smithi057 ceph-mon[30361]: pgmap v97: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-12T03:09:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:05 smithi204 ceph-mon[35013]: pgmap v97: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-12T03:09:05.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:05 smithi181 ceph-mon[35320]: pgmap v97: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 245 MiB/s wr, 1.10k op/s 2024-09-12T03:09:06.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:06 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:06.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:06 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:06.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:06 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.467+0000 7f9f53e00640 1 -- 172.21.15.57:0/493336227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 msgr2=0x7f9f4c100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.467+0000 7f9f53e00640 1 --2- 172.21.15.57:0/493336227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c100e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 -- 172.21.15.57:0/493336227 shutdown_connections 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 --2- 172.21.15.57:0/493336227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 0x7f9f4c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 --2- 172.21.15.57:0/493336227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f4c1013b0 0x7f9f4c105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 --2- 172.21.15.57:0/493336227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c100e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 -- 172.21.15.57:0/493336227 >> 172.21.15.57:0/493336227 conn(0x7f9f4c0fc6b0 msgr2=0x7f9f4c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.468+0000 7f9f53e00640 1 -- 172.21.15.57:0/493336227 shutdown_connections 2024-09-12T03:09:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.469+0000 7f9f53e00640 1 -- 172.21.15.57:0/493336227 wait complete. 2024-09-12T03:09:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.470+0000 7f9f53e00640 1 Processor -- start 2024-09-12T03:09:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.470+0000 7f9f53e00640 1 -- start start 2024-09-12T03:09:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.472+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f4c1013b0 0x7f9f4c1a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 0x7f9f4c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f53e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c118560 con 0x7f9f4c1013b0 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f53e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1183e0 con 0x7f9f4c100a90 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f53e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1186e0 con 0x7f9f4c105f10 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f51400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f51400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35054/0 (socket says 172.21.15.57:35054) 2024-09-12T03:09:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f51400640 1 -- 172.21.15.57:0/3968890642 learned_addr learned my addr 172.21.15.57:0/3968890642 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.475+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 0x7f9f4c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 -- 172.21.15.57:0/3968890642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 msgr2=0x7f9f4c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 0x7f9f4c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 -- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f4c1013b0 msgr2=0x7f9f4c1a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f4c1013b0 0x7f9f4c1a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 -- 172.21.15.57:0/3968890642 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c19ed50 con 0x7f9f4c100a90 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f51400640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009850 tx=0x7f9f40030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.476+0000 7f9f4a000640 1 -- 172.21.15.57:0/3968890642 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4003f070 con 0x7f9f4c100a90 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.477+0000 7f9f4a000640 1 -- 172.21.15.57:0/3968890642 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40004460 con 0x7f9f4c100a90 2024-09-12T03:09:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.477+0000 7f9f4a000640 1 -- 172.21.15.57:0/3968890642 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40035030 con 0x7f9f4c100a90 2024-09-12T03:09:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.477+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c19efe0 con 0x7f9f4c100a90 2024-09-12T03:09:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.477+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1ab2e0 con 0x7f9f4c100a90 2024-09-12T03:09:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.481+0000 7f9f4a000640 1 -- 172.21.15.57:0/3968890642 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f9f40042070 con 0x7f9f4c100a90 2024-09-12T03:09:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.483+0000 7f9f4a000640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 0x7f9f340a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.484+0000 7f9f4a000640 1 -- 172.21.15.57:0/3968890642 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9f400f0450 con 0x7f9f4c100a90 2024-09-12T03:09:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.484+0000 7f9f56c79740 1 Processor -- start 2024-09-12T03:09:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 -- start start 2024-09-12T03:09:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cb1e0 0x55cd855cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cd85531100 con 0x55cd855cbb00 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cd854d91b0 con 0x55cd855d63a0 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f56c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd8552d0c0 con 0x55cd855cb1e0 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35066/0 (socket says 172.21.15.57:35066) 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.485+0000 7f9f51e00640 1 -- 172.21.15.57:0/3952967281 learned_addr learned my addr 172.21.15.57:0/3952967281 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51400640 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cb1e0 0x55cd855cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51e00640 1 -- 172.21.15.57:0/3952967281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cb1e0 msgr2=0x55cd855cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cb1e0 0x55cd855cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51e00640 1 -- 172.21.15.57:0/3952967281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 msgr2=0x55cd855d5e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.486+0000 7f9f51e00640 1 -- 172.21.15.57:0/3952967281 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd85244210 con 0x55cd855d63a0 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.487+0000 7f9f50a00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 0x7f9f340a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.488+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400a430 tx=0x7f9f44009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581046d248f39147 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.488+0000 7f9f3e600640 1 -- 172.21.15.57:0/3952967281 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013790 con 0x55cd855d63a0 2024-09-12T03:09:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.488+0000 7f9f3e600640 1 -- 172.21.15.57:0/3952967281 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44019070 con 0x55cd855d63a0 2024-09-12T03:09:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.488+0000 7f9f3e600640 1 -- 172.21.15.57:0/3952967281 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44014e40 con 0x55cd855d63a0 2024-09-12T03:09:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 -- 172.21.15.57:0/3952967281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 msgr2=0x55cd855d8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400a430 tx=0x7f9f44009650 comp rx=0 tx=0).stop 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f50a00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 0x7f9f340a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005d00 tx=0x7f9f38005c50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 -- 172.21.15.57:0/3952967281 shutdown_connections 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d8790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3952967281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cb1e0 0x55cd855cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 -- 172.21.15.57:0/3952967281 >> 172.21.15.57:0/3952967281 conn(0x55cd855c7750 msgr2=0x55cd855c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.489+0000 7f9f56c79740 1 -- 172.21.15.57:0/3952967281 shutdown_connections 2024-09-12T03:09:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.490+0000 7f9f56c79740 1 -- 172.21.15.57:0/3952967281 wait complete. 2024-09-12T03:09:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 -- start start 2024-09-12T03:09:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 Processor -- start 2024-09-12T03:09:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cb1e0 0x55cd855d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cd855d8f70 con 0x55cd855cbb00 2024-09-12T03:09:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cd854ba090 con 0x55cd855cb1e0 2024-09-12T03:09:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.491+0000 7f9f56c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cd855d90f0 con 0x55cd855d63a0 2024-09-12T03:09:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.492+0000 7f9f56c79740 10 client.0 _ll_register_callbacks cb 0x55cd8550ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.492+0000 7f9f50a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.492+0000 7f9f50a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46082/0 (socket says 172.21.15.57:46082) 2024-09-12T03:09:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.492+0000 7f9f50a00640 1 -- 172.21.15.57:0/2022042417 learned_addr learned my addr 172.21.15.57:0/2022042417 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f51400640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cb1e0 0x55cd855d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f51e00640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f50a00640 1 -- 172.21.15.57:0/2022042417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 msgr2=0x55cd855d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f50a00640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f50a00640 1 -- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cb1e0 msgr2=0x55cd855d3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f50a00640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cb1e0 0x55cd855d3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.493+0000 7f9f50a00640 1 -- 172.21.15.57:0/2022042417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd85310620 con 0x55cd855cbb00 2024-09-12T03:09:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.494+0000 7f9f50a00640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f380042e0 tx=0x7f9f38005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4722867f517a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.494+0000 7f9f32a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:09:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f38005220 con 0x55cd855cbb00 2024-09-12T03:09:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9f380053c0 con 0x55cd855cbb00 2024-09-12T03:09:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f44019070 con 0x55cd855cbb00 2024-09-12T03:09:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f3802c050 con 0x55cd855cbb00 2024-09-12T03:09:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f56c79740 10 client.15405 fetch_fsmap learned FSMap version 29 2024-09-12T03:09:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.495+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd855f5d00 con 0x55cd855cbb00 2024-09-12T03:09:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.496+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44019070 con 0x55cd855cbb00 2024-09-12T03:09:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9f38018070 con 0x55cd855cbb00 2024-09-12T03:09:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 20 client.15405 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd855f7260 0x55cd855f9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 4 client.15405 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd8553e830 con 0x55cd855f7260 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.497+0000 7f9f51400640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd855f7260 0x55cd855f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.498+0000 7f9f51400640 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd855f7260 0x55cd855f9650 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e74403faaab98 server_cookie=46b8433cbbad792 in_seq=0 out_seq=0 2024-09-12T03:09:07.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.498+0000 7f9f32a00640 10 client.15405 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.500+0000 7f9f32a00640 1 -- 172.21.15.57:0/2022042417 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd8553e830 con 0x55cd855f7260 2024-09-12T03:09:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.500+0000 7f9f32a00640 10 client.15405 handle_command_reply: tid=0 2024-09-12T03:09:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 client.15405 shutdown 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 msgr2=0x55cd855d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f380042e0 tx=0x7f9f38005850 comp rx=0 tx=0).stop 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 shutdown_connections 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd855f7260 0x55cd855f9650 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cbb00 0x55cd855d3860 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 --2- 172.21.15.57:0/2022042417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cb1e0 0x55cd855d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.501+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 >> 172.21.15.57:0/2022042417 conn(0x55cd855c7750 msgr2=0x55cd855cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.502+0000 7f9f32a00640 0 client.15405 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.503+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 shutdown_connections 2024-09-12T03:09:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.503+0000 7f9f56c79740 1 -- 172.21.15.57:0/2022042417 wait complete. 2024-09-12T03:09:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.503+0000 7f9f56c79740 20 client.15405 trim_cache size 0 max 16384 2024-09-12T03:09:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.537+0000 7f9f56c79740 1 Processor -- start 2024-09-12T03:09:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.538+0000 7f9f56c79740 1 -- start start 2024-09-12T03:09:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cbb00 0x55cd855c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1af0a0 con 0x55cd855cb1e0 2024-09-12T03:09:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1af770 con 0x55cd855d63a0 2024-09-12T03:09:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.539+0000 7f9f56c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ad040 con 0x55cd855cbb00 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.540+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.540+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35088/0 (socket says 172.21.15.57:35088) 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.540+0000 7f9f51e00640 1 -- 172.21.15.57:0/1455242707 learned_addr learned my addr 172.21.15.57:0/1455242707 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.541+0000 7f9f50a00640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cbb00 0x55cd855c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.541+0000 7f9f51e00640 1 -- 172.21.15.57:0/1455242707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cbb00 msgr2=0x55cd855c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.541+0000 7f9f51e00640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cbb00 0x55cd855c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.541+0000 7f9f51e00640 1 -- 172.21.15.57:0/1455242707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 msgr2=0x55cd855c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.541+0000 7f9f51400640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.542+0000 7f9f51e00640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.542+0000 7f9f51e00640 1 -- 172.21.15.57:0/1455242707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd855d2330 con 0x55cd855d63a0 2024-09-12T03:09:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.542+0000 7f9f51400640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.543+0000 7f9f51e00640 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f44004a80 tx=0x7f9f44009c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234b91ca5f7329e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.544+0000 7f9f3dc00640 1 -- 172.21.15.57:0/1455242707 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013850 con 0x55cd855d63a0 2024-09-12T03:09:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.544+0000 7f9f3dc00640 1 -- 172.21.15.57:0/1455242707 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44019040 con 0x55cd855d63a0 2024-09-12T03:09:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.544+0000 7f9f3dc00640 1 -- 172.21.15.57:0/1455242707 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44029e00 con 0x55cd855d63a0 2024-09-12T03:09:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.545+0000 7f9f56c79740 1 -- 172.21.15.57:0/1455242707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 msgr2=0x55cd855d5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.545+0000 7f9f56c79740 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f44004a80 tx=0x7f9f44009c70 comp rx=0 tx=0).stop 2024-09-12T03:09:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.546+0000 7f9f56c79740 1 -- 172.21.15.57:0/1455242707 shutdown_connections 2024-09-12T03:09:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.546+0000 7f9f56c79740 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855d63a0 0x55cd855d5290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.546+0000 7f9f56c79740 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855cbb00 0x55cd855c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.546+0000 7f9f56c79740 1 --2- 172.21.15.57:0/1455242707 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.546+0000 7f9f56c79740 1 -- 172.21.15.57:0/1455242707 >> 172.21.15.57:0/1455242707 conn(0x55cd855c7750 msgr2=0x55cd855d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.547+0000 7f9f56c79740 1 -- 172.21.15.57:0/1455242707 shutdown_connections 2024-09-12T03:09:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.547+0000 7f9f56c79740 1 -- 172.21.15.57:0/1455242707 wait complete. 2024-09-12T03:09:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.547+0000 7f9f56c79740 1 -- start start 2024-09-12T03:09:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.548+0000 7f9f56c79740 1 Processor -- start 2024-09-12T03:09:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.549+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855de600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.549+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cbb00 0x55cd855deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f56c79740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f56c79740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ab500 con 0x55cd855cb1e0 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f56c79740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1ac970 con 0x55cd855cbb00 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f56c79740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1abbd0 con 0x55cd855d63a0 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48294/0 (socket says 172.21.15.57:48294) 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 -- 172.21.15.57:0/3102951607 learned_addr learned my addr 172.21.15.57:0/3102951607 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f56c79740 10 client.0 _ll_register_callbacks cb 0x7f9f38010f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f50a00640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cbb00 0x55cd855deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 -- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cbb00 msgr2=0x55cd855deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cbb00 0x55cd855deb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 -- 172.21.15.57:0/3102951607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 msgr2=0x55cd855de600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.550+0000 7f9f51e00640 1 -- 172.21.15.57:0/3102951607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd855d0590 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.551+0000 7f9f51e00640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009690 tx=0x7f9f44029c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53ad30584fce9f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.551+0000 7f9f32a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.552+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44013850 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.552+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f44019040 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.552+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f9f44013650 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.552+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f44013850 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.552+0000 7f9f56c79740 10 client.25507 fetch_fsmap learned FSMap version 29 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd85661a70 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44019040 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9f44014ad0 con 0x55cd855d63a0 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f56c79740 10 client.25507 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:09:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f56c79740 10 client.25507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f56c79740 20 client.25507 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.553+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd85661ea0 0x55cd856642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.554+0000 7f9f56c79740 4 client.25507 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.554+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd85538b80 con 0x55cd85661ea0 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.554+0000 7f9f51400640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd85661ea0 0x55cd856642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.555+0000 7f9f51400640 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd85661ea0 0x55cd856642e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced7d2d5bb4996df server_cookie=78fae98b5e4622 in_seq=0 out_seq=0 2024-09-12T03:09:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.555+0000 7f9f32a00640 10 client.25507 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.576+0000 7f9f32a00640 1 -- 172.21.15.57:0/3102951607 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd85538b80 con 0x55cd85661ea0 2024-09-12T03:09:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.576+0000 7f9f32a00640 10 client.25507 handle_command_reply: tid=0 2024-09-12T03:09:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.576+0000 7f9f56c79740 1 client.25507 shutdown 2024-09-12T03:09:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 msgr2=0x55cd855cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009690 tx=0x7f9f44029c90 comp rx=0 tx=0).stop 2024-09-12T03:09:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 shutdown_connections 2024-09-12T03:09:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cd85661ea0 0x55cd856642e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cd855d63a0 0x55cd855cfd60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cd855cbb00 0x55cd855deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 --2- 172.21.15.57:0/3102951607 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cd855cb1e0 0x55cd855de600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 >> 172.21.15.57:0/3102951607 conn(0x55cd855c7750 msgr2=0x55cd855d3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.580+0000 7f9f32a00640 0 client.25507 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.581+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 shutdown_connections 2024-09-12T03:09:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.581+0000 7f9f56c79740 1 -- 172.21.15.57:0/3102951607 wait complete. 2024-09-12T03:09:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.581+0000 7f9f56c79740 20 client.25507 trim_cache size 0 max 16384 2024-09-12T03:09:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:09:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:09:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:09:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:09:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 msgr2=0x7f9f340a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 0x7f9f340a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f38005d00 tx=0x7f9f38005c50 comp rx=0 tx=0).stop 2024-09-12T03:09:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 msgr2=0x7f9f4c1a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009850 tx=0x7f9f40030650 comp rx=0 tx=0).stop 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 shutdown_connections 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4c105f10 0x7f9f4c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f9f340a2e20 0x7f9f340a52e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9f4c1013b0 0x7f9f4c1a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 --2- 172.21.15.57:0/3968890642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f4c100a90 0x7f9f4c1a4420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.590+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 >> 172.21.15.57:0/3968890642 conn(0x7f9f4c0fc6b0 msgr2=0x7f9f4c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.591+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 shutdown_connections 2024-09-12T03:09:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:07.593+0000 7f9f53e00640 1 -- 172.21.15.57:0/3968890642 wait complete. 2024-09-12T03:09:07.874 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:07 smithi057 ceph-mon[30361]: pgmap v98: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 210 MiB/s wr, 954 op/s 2024-09-12T03:09:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:07 smithi204 ceph-mon[35013]: pgmap v98: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 210 MiB/s wr, 954 op/s 2024-09-12T03:09:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:07 smithi181 ceph-mon[35320]: pgmap v98: 129 pgs: 129 active+clean; 17 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 210 MiB/s wr, 954 op/s 2024-09-12T03:09:08.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:08 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:08.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:08 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:08.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:08 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:09:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:08 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:08 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:08 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:09:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:08 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:08 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:09:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:08 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:09:09.884 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:09 smithi057 ceph-mon[30361]: pgmap v99: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 314 MiB/s wr, 1.57k op/s 2024-09-12T03:09:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:09 smithi204 ceph-mon[35013]: pgmap v99: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 314 MiB/s wr, 1.57k op/s 2024-09-12T03:09:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:09 smithi181 ceph-mon[35320]: pgmap v99: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 314 MiB/s wr, 1.57k op/s 2024-09-12T03:09:11.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:11 smithi057 ceph-mon[30361]: pgmap v100: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 207 MiB/s wr, 1.11k op/s 2024-09-12T03:09:11.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:11 smithi204 ceph-mon[35013]: pgmap v100: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 207 MiB/s wr, 1.11k op/s 2024-09-12T03:09:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:11 smithi181 ceph-mon[35320]: pgmap v100: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 207 MiB/s wr, 1.11k op/s 2024-09-12T03:09:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:13 smithi057 ceph-mon[30361]: pgmap v101: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 254 MiB/s wr, 1.50k op/s 2024-09-12T03:09:13.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:13 smithi204 ceph-mon[35013]: pgmap v101: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 254 MiB/s wr, 1.50k op/s 2024-09-12T03:09:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:13 smithi181 ceph-mon[35320]: pgmap v101: 129 pgs: 129 active+clean; 18 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 254 MiB/s wr, 1.50k op/s 2024-09-12T03:09:15.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:15 smithi057 ceph-mon[30361]: pgmap v102: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 207 MiB/s wr, 1.37k op/s 2024-09-12T03:09:15.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:15 smithi204 ceph-mon[35013]: pgmap v102: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 207 MiB/s wr, 1.37k op/s 2024-09-12T03:09:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:15 smithi181 ceph-mon[35320]: pgmap v102: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 207 MiB/s wr, 1.37k op/s 2024-09-12T03:09:17.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:17 smithi057 ceph-mon[30361]: pgmap v103: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 174 MiB/s wr, 1.20k op/s 2024-09-12T03:09:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:17 smithi204 ceph-mon[35013]: pgmap v103: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 174 MiB/s wr, 1.20k op/s 2024-09-12T03:09:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:17 smithi181 ceph-mon[35320]: pgmap v103: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 174 MiB/s wr, 1.20k op/s 2024-09-12T03:09:18.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:18 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:18.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:18 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:18 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:09:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:19 smithi057 ceph-mon[30361]: pgmap v104: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 215 MiB/s wr, 1.77k op/s 2024-09-12T03:09:19.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:19 smithi204 ceph-mon[35013]: pgmap v104: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 215 MiB/s wr, 1.77k op/s 2024-09-12T03:09:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:19 smithi181 ceph-mon[35320]: pgmap v104: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 215 MiB/s wr, 1.77k op/s 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 -- 172.21.15.57:0/1014232681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4330006940 msgr2=0x7f4330006d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 --2- 172.21.15.57:0/1014232681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4330006940 0x7f4330006d20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f432c009900 tx=0x7f432c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 -- 172.21.15.57:0/1014232681 shutdown_connections 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 --2- 172.21.15.57:0/1014232681 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f43300a2060 0x7f43300a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 --2- 172.21.15.57:0/1014232681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4330007260 0x7f43300a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 --2- 172.21.15.57:0/1014232681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4330006940 0x7f4330006d20 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f432c009900 tx=0x7f432c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.853+0000 7f433ca00640 1 -- 172.21.15.57:0/1014232681 >> 172.21.15.57:0/1014232681 conn(0x7f433000ca00 msgr2=0x7f433000ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.854+0000 7f433ca00640 1 -- 172.21.15.57:0/1014232681 shutdown_connections 2024-09-12T03:09:20.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.854+0000 7f433ca00640 1 -- 172.21.15.57:0/1014232681 wait complete. 2024-09-12T03:09:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.856+0000 7f433ca00640 1 Processor -- start 2024-09-12T03:09:20.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.857+0000 7f433ca00640 1 -- start start 2024-09-12T03:09:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4330007260 0x7f433009ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f433009f650 0x7f433013e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f43300a7c40 con 0x7f433009f650 2024-09-12T03:09:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f43300a7ac0 con 0x7f4330007260 2024-09-12T03:09:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.860+0000 7f433ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43300a7dc0 con 0x7f43300a2060 2024-09-12T03:09:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.861+0000 7f4336a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.861+0000 7f4336a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:40840/0 (socket says 172.21.15.57:40840) 2024-09-12T03:09:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.861+0000 7f4336a00640 1 -- 172.21.15.57:0/277589846 learned_addr learned my addr 172.21.15.57:0/277589846 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.861+0000 7f4337e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f433009f650 0x7f433013e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.862+0000 7f4336a00640 1 -- 172.21.15.57:0/277589846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4330007260 msgr2=0x7f433009ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.862+0000 7f4336a00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4330007260 0x7f433009ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.862+0000 7f4336a00640 1 -- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f433009f650 msgr2=0x7f433013e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.862+0000 7f4336a00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f433009f650 0x7f433013e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.862+0000 7f4336a00640 1 -- 172.21.15.57:0/277589846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433013eea0 con 0x7f43300a2060 2024-09-12T03:09:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.863+0000 7f4336a00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f432400c890 tx=0x7f432400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.863+0000 7f432be00640 1 -- 172.21.15.57:0/277589846 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324004220 con 0x7f43300a2060 2024-09-12T03:09:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.864+0000 7f432be00640 1 -- 172.21.15.57:0/277589846 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43240043c0 con 0x7f43300a2060 2024-09-12T03:09:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.864+0000 7f432be00640 1 -- 172.21.15.57:0/277589846 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324004b10 con 0x7f43300a2060 2024-09-12T03:09:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.864+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433013f070 con 0x7f43300a2060 2024-09-12T03:09:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.866+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433013f4e0 con 0x7f43300a2060 2024-09-12T03:09:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.872+0000 7f432be00640 1 -- 172.21.15.57:0/277589846 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f4324004cb0 con 0x7f43300a2060 2024-09-12T03:09:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.875+0000 7f4340fc3740 1 Processor -- start 2024-09-12T03:09:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 -- start start 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4b7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555f7d41d100 con 0x555f7d4b73b0 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7d3c51b0 con 0x555f7d4c2570 2024-09-12T03:09:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.876+0000 7f4340fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f7d4190c0 con 0x555f7d4b7cd0 2024-09-12T03:09:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.878+0000 7f4336a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.878+0000 7f4336a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46350/0 (socket says 172.21.15.57:46350) 2024-09-12T03:09:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.878+0000 7f4336a00640 1 -- 172.21.15.57:0/2059990564 learned_addr learned my addr 172.21.15.57:0/2059990564 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.878+0000 7f4337e00640 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.879+0000 7f4337e00640 1 -- 172.21.15.57:0/2059990564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 msgr2=0x555f7d4c2030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.879+0000 7f4337e00640 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c2030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.879+0000 7f4337e00640 1 -- 172.21.15.57:0/2059990564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 msgr2=0x555f7d4c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.879+0000 7f4337e00640 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.879+0000 7f4337e00640 1 -- 172.21.15.57:0/2059990564 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7d130210 con 0x555f7d4b73b0 2024-09-12T03:09:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.880+0000 7f4337e00640 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4b7790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4338067f00 tx=0x7f433806b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cf9d7309991d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.881+0000 7f431fe00640 1 -- 172.21.15.57:0/2059990564 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338069d50 con 0x555f7d4b73b0 2024-09-12T03:09:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.881+0000 7f431fe00640 1 -- 172.21.15.57:0/2059990564 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433806bba0 con 0x555f7d4b73b0 2024-09-12T03:09:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.881+0000 7f431fe00640 1 -- 172.21.15.57:0/2059990564 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380745d0 con 0x555f7d4b73b0 2024-09-12T03:09:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.882+0000 7f4340fc3740 1 -- 172.21.15.57:0/2059990564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 msgr2=0x555f7d4b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.882+0000 7f4340fc3740 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4b7790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4338067f00 tx=0x7f433806b420 comp rx=0 tx=0).stop 2024-09-12T03:09:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.895+0000 7f4340fc3740 1 -- 172.21.15.57:0/2059990564 shutdown_connections 2024-09-12T03:09:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.896+0000 7f4340fc3740 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.896+0000 7f4340fc3740 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.896+0000 7f4340fc3740 1 --2- 172.21.15.57:0/2059990564 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4b7790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.896+0000 7f4340fc3740 1 -- 172.21.15.57:0/2059990564 >> 172.21.15.57:0/2059990564 conn(0x555f7d4b3920 msgr2=0x555f7d4b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.898+0000 7f4340fc3740 1 -- 172.21.15.57:0/2059990564 shutdown_connections 2024-09-12T03:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.899+0000 7f4340fc3740 1 -- 172.21.15.57:0/2059990564 wait complete. 2024-09-12T03:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.899+0000 7f432be00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 0x7f43200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.899+0000 7f432be00640 1 -- 172.21.15.57:0/277589846 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f43240cf780 con 0x7f43300a2060 2024-09-12T03:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.900+0000 7f4340fc3740 1 -- start start 2024-09-12T03:09:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.900+0000 7f4340fc3740 1 Processor -- start 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.901+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 0x555f7d4bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.901+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.901+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.901+0000 7f4340fc3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555f7d4d6150 con 0x555f7d4b7cd0 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.901+0000 7f4340fc3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555f7d3a6090 con 0x555f7d4c2570 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.902+0000 7f4340fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f7d4d62d0 con 0x555f7d4b73b0 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.902+0000 7f4340fc3740 10 client.0 _ll_register_callbacks cb 0x555f7d3faf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.902+0000 7f4337400640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 0x7f43200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.904+0000 7f4337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.905+0000 7f4337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46358/0 (socket says 172.21.15.57:46358) 2024-09-12T03:09:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.905+0000 7f4337e00640 1 -- 172.21.15.57:0/3046581472 learned_addr learned my addr 172.21.15.57:0/3046581472 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.905+0000 7f4336a00640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.905+0000 7f4337400640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 0x555f7d4bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 -- 172.21.15.57:0/3046581472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 msgr2=0x555f7d4bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 0x555f7d4bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 -- 172.21.15.57:0/3046581472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 msgr2=0x555f7d4bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 -- 172.21.15.57:0/3046581472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7d1fc620 con 0x555f7d4c2570 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337400640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 0x555f7d4bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337400640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 0x7f43200a5230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c000bc0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4336a00640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f431cc00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:09:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.906+0000 7f4337e00640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4338067ad0 tx=0x7f433806c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8335957658fae83d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.907+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433806ca30 con 0x555f7d4c2570 2024-09-12T03:09:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.907+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f4338069d50 con 0x555f7d4c2570 2024-09-12T03:09:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.907+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433806bba0 con 0x555f7d4c2570 2024-09-12T03:09:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.908+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4338068b70 con 0x555f7d4c2570 2024-09-12T03:09:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.908+0000 7f4340fc3740 10 client.25487 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.908+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f7d4f2e00 con 0x555f7d4c2570 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.909+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433806bba0 con 0x555f7d4c2570 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4338084020 con 0x555f7d4c2570 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 10 client.25487 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 20 client.25487 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d4f42c0 0x555f7d4f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 4 client.25487 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.911+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7d42a830 con 0x555f7d4f42c0 2024-09-12T03:09:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.912+0000 7f4337400640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d4f42c0 0x555f7d4f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.913+0000 7f4337400640 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d4f42c0 0x555f7d4f66b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2791257eb65ce3b server_cookie=e7d725169c2efbf3 in_seq=0 out_seq=0 2024-09-12T03:09:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.913+0000 7f431cc00640 10 client.25487 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.914+0000 7f431cc00640 1 -- 172.21.15.57:0/3046581472 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f7d42a830 con 0x555f7d4f42c0 2024-09-12T03:09:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.914+0000 7f431cc00640 10 client.25487 handle_command_reply: tid=0 2024-09-12T03:09:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.914+0000 7f4340fc3740 1 client.25487 shutdown 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.918+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 msgr2=0x555f7d4bfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.918+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4338067ad0 tx=0x7f433806c8a0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 shutdown_connections 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d4f42c0 0x555f7d4f66b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4bfeb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3046581472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b73b0 0x555f7d4bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 >> 172.21.15.57:0/3046581472 conn(0x555f7d4b3920 msgr2=0x555f7d4b9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.919+0000 7f431cc00640 0 client.25487 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.921+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 shutdown_connections 2024-09-12T03:09:20.922 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.923+0000 7f4340fc3740 1 -- 172.21.15.57:0/3046581472 wait complete. 2024-09-12T03:09:20.923 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.923+0000 7f4340fc3740 20 client.25487 trim_cache size 0 max 16384 2024-09-12T03:09:20.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.965+0000 7f4340fc3740 1 Processor -- start 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 -- start start 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 0x555f7d4fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4330143890 con 0x555f7d4b7cd0 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4330141160 con 0x555f7d4b73b0 2024-09-12T03:09:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.969+0000 7f4340fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f43300a0100 con 0x555f7d4c2570 2024-09-12T03:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.970+0000 7f4337400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.970+0000 7f4337400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46372/0 (socket says 172.21.15.57:46372) 2024-09-12T03:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.970+0000 7f4337400640 1 -- 172.21.15.57:0/3172182614 learned_addr learned my addr 172.21.15.57:0/3172182614 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 0x555f7d4fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 -- 172.21.15.57:0/3172182614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 msgr2=0x555f7d4ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 -- 172.21.15.57:0/3172182614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 msgr2=0x555f7d4c1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.971+0000 7f4337e00640 1 -- 172.21.15.57:0/3172182614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7d4b3750 con 0x555f7d4c2570 2024-09-12T03:09:20.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.972+0000 7f4337400640 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.972+0000 7f4337e00640 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 0x555f7d4fbe20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4338050d60 tx=0x7f4338063640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab9fd63c66e7c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.973+0000 7f431f400640 1 -- 172.21.15.57:0/3172182614 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433806c330 con 0x555f7d4c2570 2024-09-12T03:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.973+0000 7f431f400640 1 -- 172.21.15.57:0/3172182614 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433806c910 con 0x555f7d4c2570 2024-09-12T03:09:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.973+0000 7f431f400640 1 -- 172.21.15.57:0/3172182614 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433808f3d0 con 0x555f7d4c2570 2024-09-12T03:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 -- 172.21.15.57:0/3172182614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 msgr2=0x555f7d4fbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 0x555f7d4fbe20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4338050d60 tx=0x7f4338063640 comp rx=0 tx=0).stop 2024-09-12T03:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 -- 172.21.15.57:0/3172182614 shutdown_connections 2024-09-12T03:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4c2570 0x555f7d4fbe20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b7cd0 0x555f7d4c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3172182614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4b73b0 0x555f7d4ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.974+0000 7f4340fc3740 1 -- 172.21.15.57:0/3172182614 >> 172.21.15.57:0/3172182614 conn(0x555f7d4b3920 msgr2=0x555f7d4c0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.975+0000 7f4340fc3740 1 -- 172.21.15.57:0/3172182614 shutdown_connections 2024-09-12T03:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.975+0000 7f4340fc3740 1 -- 172.21.15.57:0/3172182614 wait complete. 2024-09-12T03:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.975+0000 7f4340fc3740 1 -- start start 2024-09-12T03:09:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.976+0000 7f4340fc3740 1 Processor -- start 2024-09-12T03:09:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4d8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4d8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f433013f7d0 con 0x555f7d4b73b0 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4330140a90 con 0x555f7d4c2570 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f433013fcf0 con 0x555f7d4b7cd0 2024-09-12T03:09:20.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.977+0000 7f4340fc3740 10 client.0 _ll_register_callbacks cb 0x7f4338050fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.978+0000 7f4336a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4d8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.978+0000 7f4337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.978+0000 7f4337e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4d8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46378/0 (socket says 172.21.15.57:46378) 2024-09-12T03:09:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.978+0000 7f4337e00640 1 -- 172.21.15.57:0/3295266876 learned_addr learned my addr 172.21.15.57:0/3295266876 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.980+0000 7f4336a00640 1 -- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 msgr2=0x555f7d4d8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.980+0000 7f4336a00640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4d8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.980+0000 7f4336a00640 1 -- 172.21.15.57:0/3295266876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 msgr2=0x555f7d4d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.981+0000 7f4337400640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.981+0000 7f4336a00640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.983+0000 7f4336a00640 1 -- 172.21.15.57:0/3295266876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7d55cff0 con 0x555f7d4b7cd0 2024-09-12T03:09:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.984+0000 7f4337400640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.984+0000 7f4336a00640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4d8ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x555f7d4b9f50 tx=0x7f4324005530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad3697990175f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.984+0000 7f431cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.985+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240b4060 con 0x555f7d4b7cd0 2024-09-12T03:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.985+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f4324097fe0 con 0x555f7d4b7cd0 2024-09-12T03:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.985+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433806c910 con 0x555f7d4b7cd0 2024-09-12T03:09:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.985+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43240b9050 con 0x555f7d4b7cd0 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.985+0000 7f4340fc3740 10 client.25519 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f7d55e340 con 0x555f7d4b7cd0 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433806c910 con 0x555f7d4b7cd0 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f43240cf350 con 0x555f7d4b7cd0 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 10 client.25519 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 10 client.25519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 20 client.25519 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d55ea10 0x555f7d560e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 4 client.25519 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:09:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.986+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f7d424b80 con 0x555f7d55ea10 2024-09-12T03:09:20.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.987+0000 7f4337400640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d55ea10 0x555f7d560e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.989+0000 7f4337400640 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d55ea10 0x555f7d560e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab513e2a186aebb server_cookie=887f4f371a586274 in_seq=0 out_seq=0 2024-09-12T03:09:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:20.990+0000 7f431cc00640 10 client.25519 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.432+0000 7f431cc00640 1 -- 172.21.15.57:0/3295266876 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f7d424b80 con 0x555f7d55ea10 2024-09-12T03:09:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.432+0000 7f431cc00640 10 client.25519 handle_command_reply: tid=0 2024-09-12T03:09:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.432+0000 7f4340fc3740 1 client.25519 shutdown 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.432+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 msgr2=0x555f7d4d8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.432+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4d8ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x555f7d4b9f50 tx=0x7f4324005530 comp rx=0 tx=0).stop 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 shutdown_connections 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555f7d55ea10 0x555f7d560e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555f7d4c2570 0x555f7d4d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f7d4b7cd0 0x555f7d4d8ab0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 --2- 172.21.15.57:0/3295266876 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555f7d4b73b0 0x555f7d4d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f431cc00640 0 client.25519 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.433+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 >> 172.21.15.57:0/3295266876 conn(0x555f7d4b3920 msgr2=0x555f7d4b9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.434+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 shutdown_connections 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.434+0000 7f4340fc3740 1 -- 172.21.15.57:0/3295266876 wait complete. 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.434+0000 7f4340fc3740 20 client.25519 trim_cache size 0 max 16384 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:09:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 msgr2=0x7f43200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 0x7f43200a5230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f432c009230 tx=0x7f432c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 msgr2=0x7f433009f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f432400c890 tx=0x7f432400cd50 comp rx=0 tx=0).stop 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 shutdown_connections 2024-09-12T03:09:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f43200a2d70 0x7f43200a5230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f433009f650 0x7f433013e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43300a2060 0x7f433009f110 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 --2- 172.21.15.57:0/277589846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4330007260 0x7f433009ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 >> 172.21.15.57:0/277589846 conn(0x7f433000ca00 msgr2=0x7f4330093af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 shutdown_connections 2024-09-12T03:09:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:21.436+0000 7f433ca00640 1 -- 172.21.15.57:0/277589846 wait complete. 2024-09-12T03:09:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:21 smithi057 ceph-mon[30361]: pgmap v105: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 1.27k op/s 2024-09-12T03:09:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:21 smithi204 ceph-mon[35013]: pgmap v105: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 1.27k op/s 2024-09-12T03:09:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:21 smithi181 ceph-mon[35320]: pgmap v105: 129 pgs: 129 active+clean; 18 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 119 MiB/s wr, 1.27k op/s 2024-09-12T03:09:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:23 smithi057 ceph-mon[30361]: pgmap v106: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 217 MiB/s wr, 2.17k op/s 2024-09-12T03:09:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:23 smithi204 ceph-mon[35013]: pgmap v106: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 217 MiB/s wr, 2.17k op/s 2024-09-12T03:09:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:23 smithi181 ceph-mon[35320]: pgmap v106: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 217 MiB/s wr, 2.17k op/s 2024-09-12T03:09:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:25 smithi057 ceph-mon[30361]: pgmap v107: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 216 MiB/s wr, 2.22k op/s 2024-09-12T03:09:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:25 smithi204 ceph-mon[35013]: pgmap v107: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 216 MiB/s wr, 2.22k op/s 2024-09-12T03:09:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:25 smithi181 ceph-mon[35320]: pgmap v107: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 216 MiB/s wr, 2.22k op/s 2024-09-12T03:09:27.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:26 smithi057 ceph-mon[30361]: pgmap v108: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 193 MiB/s wr, 2.02k op/s 2024-09-12T03:09:27.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:26 smithi204 ceph-mon[35013]: pgmap v108: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 193 MiB/s wr, 2.02k op/s 2024-09-12T03:09:27.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:26 smithi181 ceph-mon[35320]: pgmap v108: 129 pgs: 129 active+clean; 19 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 193 MiB/s wr, 2.02k op/s 2024-09-12T03:09:29.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:28 smithi181 ceph-mon[35320]: pgmap v109: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 2.44k op/s 2024-09-12T03:09:29.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:28 smithi057 ceph-mon[30361]: pgmap v109: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 2.44k op/s 2024-09-12T03:09:29.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:28 smithi204 ceph-mon[35013]: pgmap v109: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 227 MiB/s wr, 2.44k op/s 2024-09-12T03:09:31.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:30 smithi181 ceph-mon[35320]: pgmap v110: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 205 MiB/s wr, 2.06k op/s 2024-09-12T03:09:31.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:30 smithi057 ceph-mon[30361]: pgmap v110: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 205 MiB/s wr, 2.06k op/s 2024-09-12T03:09:31.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:30 smithi204 ceph-mon[35013]: pgmap v110: 129 pgs: 129 active+clean; 19 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 205 MiB/s wr, 2.06k op/s 2024-09-12T03:09:33.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:32 smithi057 ceph-mon[30361]: pgmap v111: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 260 MiB/s wr, 2.52k op/s 2024-09-12T03:09:33.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:32 smithi204 ceph-mon[35013]: pgmap v111: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 260 MiB/s wr, 2.52k op/s 2024-09-12T03:09:33.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:32 smithi181 ceph-mon[35320]: pgmap v111: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 260 MiB/s wr, 2.52k op/s 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.751+0000 7fd88b800640 1 -- 172.21.15.57:0/1239783892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 msgr2=0x7fd884114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.751+0000 7fd88b800640 1 --2- 172.21.15.57:0/1239783892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd884114570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd874009ca0 tx=0x7fd87402a8d0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.752+0000 7fd88b800640 1 -- 172.21.15.57:0/1239783892 shutdown_connections 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.752+0000 7fd88b800640 1 --2- 172.21.15.57:0/1239783892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd884114ab0 0x7fd884116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.752+0000 7fd88b800640 1 --2- 172.21.15.57:0/1239783892 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd884114570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.752+0000 7fd88b800640 1 --2- 172.21.15.57:0/1239783892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8841091f0 0x7fd8841095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.752+0000 7fd88b800640 1 -- 172.21.15.57:0/1239783892 >> 172.21.15.57:0/1239783892 conn(0x7fd88406ce00 msgr2=0x7fd88406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.753+0000 7fd88b800640 1 -- 172.21.15.57:0/1239783892 shutdown_connections 2024-09-12T03:09:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.753+0000 7fd88b800640 1 -- 172.21.15.57:0/1239783892 wait complete. 2024-09-12T03:09:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.755+0000 7fd88b800640 1 Processor -- start 2024-09-12T03:09:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.757+0000 7fd88b800640 1 -- start start 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.758+0000 7fd88b800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 0x7fd88410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.758+0000 7fd88b800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd88410a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.758+0000 7fd88b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 0x7fd88410ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 0x7fd88410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd88b800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd88411a770 con 0x7fd884109b10 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd88b800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd88411a5f0 con 0x7fd8841091f0 2024-09-12T03:09:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd88b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd88411a8f0 con 0x7fd884114ab0 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 0x7fd88410a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50806/0 (socket says 172.21.15.57:50806) 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd888e00640 1 -- 172.21.15.57:0/1408022646 learned_addr learned my addr 172.21.15.57:0/1408022646 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 0x7fd88410ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.759+0000 7fd87be00640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd88410a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd889800640 1 -- 172.21.15.57:0/1408022646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 msgr2=0x7fd88410a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 0x7fd88410a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd889800640 1 -- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 msgr2=0x7fd88410a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd88410a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd889800640 1 -- 172.21.15.57:0/1408022646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8841b33b0 con 0x7fd884114ab0 2024-09-12T03:09:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.760+0000 7fd87be00640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd88410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.761+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 0x7fd88410ae50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd88406b320 tx=0x7fd87c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.762+0000 7fd882000640 1 -- 172.21.15.57:0/1408022646 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c015020 con 0x7fd884114ab0 2024-09-12T03:09:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.762+0000 7fd882000640 1 -- 172.21.15.57:0/1408022646 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87c011d10 con 0x7fd884114ab0 2024-09-12T03:09:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.762+0000 7fd882000640 1 -- 172.21.15.57:0/1408022646 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c010e60 con 0x7fd884114ab0 2024-09-12T03:09:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.763+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8841b35e0 con 0x7fd884114ab0 2024-09-12T03:09:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.763+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8841b3ab0 con 0x7fd884114ab0 2024-09-12T03:09:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.768+0000 7fd88e7e8740 1 Processor -- start 2024-09-12T03:09:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.768+0000 7fd882000640 1 -- 172.21.15.57:0/1408022646 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fd87c018020 con 0x7fd884114ab0 2024-09-12T03:09:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.769+0000 7fd88e7e8740 1 -- start start 2024-09-12T03:09:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.769+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc41c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.769+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.769+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc429790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5563bc382100 con 0x5563bc4273a0 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5563bc32a1b0 con 0x5563bc41cb00 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5563bc37e0c0 con 0x5563bc41c1e0 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50822/0 (socket says 172.21.15.57:50822) 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd888e00640 1 -- 172.21.15.57:0/3892100302 learned_addr learned my addr 172.21.15.57:0/3892100302 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd87be00640 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc41c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.770+0000 7fd888e00640 1 -- 172.21.15.57:0/3892100302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 msgr2=0x5563bc41c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.772+0000 7fd882000640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 0x7fd8680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.773+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 0x7fd8680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.773+0000 7fd882000640 1 -- 172.21.15.57:0/1408022646 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd87c0cee80 con 0x7fd884114ab0 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.774+0000 7fd888e00640 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc41c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.774+0000 7fd888e00640 1 -- 172.21.15.57:0/3892100302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 msgr2=0x5563bc429790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.774+0000 7fd888e00640 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc429790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.774+0000 7fd888e00640 1 -- 172.21.15.57:0/3892100302 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bc095210 con 0x5563bc41cb00 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.774+0000 7fd87be00640 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc41c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.775+0000 7fd888e00640 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00bd90 tx=0x7fd86c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2afb04535fc6aa06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.775+0000 7fd87a000640 1 -- 172.21.15.57:0/3892100302 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c015070 con 0x5563bc41cb00 2024-09-12T03:09:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.775+0000 7fd87a000640 1 -- 172.21.15.57:0/3892100302 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c0035f0 con 0x5563bc41cb00 2024-09-12T03:09:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.775+0000 7fd87a000640 1 -- 172.21.15.57:0/3892100302 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0116a0 con 0x5563bc41cb00 2024-09-12T03:09:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.776+0000 7fd88e7e8740 1 -- 172.21.15.57:0/3892100302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 msgr2=0x5563bc426e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.776+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00bd90 tx=0x7fd86c00f730 comp rx=0 tx=0).stop 2024-09-12T03:09:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.776+0000 7fd889800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 0x7fd8680a53c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c010880 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.778+0000 7fd88e7e8740 1 -- 172.21.15.57:0/3892100302 shutdown_connections 2024-09-12T03:09:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.778+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc429790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.778+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc426e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.778+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/3892100302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc41c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.778+0000 7fd88e7e8740 1 -- 172.21.15.57:0/3892100302 >> 172.21.15.57:0/3892100302 conn(0x5563bc418750 msgr2=0x5563bc418b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.783+0000 7fd88e7e8740 1 -- 172.21.15.57:0/3892100302 shutdown_connections 2024-09-12T03:09:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.783+0000 7fd88e7e8740 1 -- 172.21.15.57:0/3892100302 wait complete. 2024-09-12T03:09:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.784+0000 7fd88e7e8740 1 -- start start 2024-09-12T03:09:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.784+0000 7fd88e7e8740 1 Processor -- start 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc424420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc424ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5563bc429f70 con 0x5563bc4273a0 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5563bc30b090 con 0x5563bc41cb00 2024-09-12T03:09:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5563bc42a0f0 con 0x5563bc41c1e0 2024-09-12T03:09:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd88e7e8740 10 client.0 _ll_register_callbacks cb 0x5563bc35ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd87be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc424420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd888e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50824/0 (socket says 172.21.15.57:50824) 2024-09-12T03:09:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.785+0000 7fd888e00640 1 -- 172.21.15.57:0/899492590 learned_addr learned my addr 172.21.15.57:0/899492590 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 -- 172.21.15.57:0/899492590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 msgr2=0x5563bc424420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc424420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 -- 172.21.15.57:0/899492590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 msgr2=0x5563bc424ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc424ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 -- 172.21.15.57:0/899492590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bc161620 con 0x5563bc41cb00 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd888e00640 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000c00 tx=0x7fd86c004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72f8d2bc8ff1bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.786+0000 7fd872a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:09:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.788+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c015070 con 0x5563bc41cb00 2024-09-12T03:09:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.788+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd86c00fde0 con 0x5563bc41cb00 2024-09-12T03:09:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.788+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c0035f0 con 0x5563bc41cb00 2024-09-12T03:09:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.788+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd86c004350 con 0x5563bc41cb00 2024-09-12T03:09:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.789+0000 7fd88e7e8740 10 client.25502 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.789+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563bc435c10 con 0x5563bc41cb00 2024-09-12T03:09:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.789+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c0035f0 con 0x5563bc41cb00 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.790+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd86c02ec80 con 0x5563bc41cb00 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 10 client.25502 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 20 client.25502 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc437080 0x5563bc439470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 4 client.25502 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563bc38f830 con 0x5563bc437080 2024-09-12T03:09:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.791+0000 7fd87be00640 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc437080 0x5563bc439470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.792+0000 7fd87be00640 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc437080 0x5563bc439470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5795116099279b2 server_cookie=371a505764b8bb9d in_seq=0 out_seq=0 2024-09-12T03:09:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.792+0000 7fd872a00640 10 client.25502 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.792+0000 7fd872a00640 1 -- 172.21.15.57:0/899492590 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563bc38f830 con 0x5563bc437080 2024-09-12T03:09:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd872a00640 10 client.25502 handle_command_reply: tid=0 2024-09-12T03:09:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 client.25502 shutdown 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 msgr2=0x5563bc424960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000c00 tx=0x7fd86c004230 comp rx=0 tx=0).stop 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 shutdown_connections 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc437080 0x5563bc439470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc424ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41cb00 0x5563bc424960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/899492590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41c1e0 0x5563bc424420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 >> 172.21.15.57:0/899492590 conn(0x5563bc418750 msgr2=0x5563bc41e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.793+0000 7fd872a00640 0 client.25502 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.794+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 shutdown_connections 2024-09-12T03:09:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.794+0000 7fd88e7e8740 1 -- 172.21.15.57:0/899492590 wait complete. 2024-09-12T03:09:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.794+0000 7fd88e7e8740 20 client.25502 trim_cache size 0 max 16384 2024-09-12T03:09:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 Processor -- start 2024-09-12T03:09:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 -- start start 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 0x5563bc424cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc4273a0 0x5563bc43f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841c0660 con 0x5563bc41c1e0 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841bdf30 con 0x5563bc4273a0 2024-09-12T03:09:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.826+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841bbd80 con 0x5563bc41cb00 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51844/0 (socket says 172.21.15.57:51844) 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 -- 172.21.15.57:0/1692809107 learned_addr learned my addr 172.21.15.57:0/1692809107 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 -- 172.21.15.57:0/1692809107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc4273a0 msgr2=0x5563bc43f070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd87be00640 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 0x5563bc424cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc4273a0 0x5563bc43f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 -- 172.21.15.57:0/1692809107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 msgr2=0x5563bc424cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 0x5563bc424cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.827+0000 7fd888e00640 1 -- 172.21.15.57:0/1692809107 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563bc41f130 con 0x5563bc41cb00 2024-09-12T03:09:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.828+0000 7fd87be00640 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 0x5563bc424cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.828+0000 7fd888e00640 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c010 tx=0x7fd86c00fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d29d63b1eb864ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.828+0000 7fd879600640 1 -- 172.21.15.57:0/1692809107 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c015070 con 0x5563bc41cb00 2024-09-12T03:09:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.828+0000 7fd879600640 1 -- 172.21.15.57:0/1692809107 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c033020 con 0x5563bc41cb00 2024-09-12T03:09:34.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.829+0000 7fd879600640 1 -- 172.21.15.57:0/1692809107 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0039b0 con 0x5563bc41cb00 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- 172.21.15.57:0/1692809107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 msgr2=0x5563bc43eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00c010 tx=0x7fd86c00fc70 comp rx=0 tx=0).stop 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- 172.21.15.57:0/1692809107 shutdown_connections 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc4273a0 0x5563bc43f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc43eb30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/1692809107 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc41c1e0 0x5563bc424cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- 172.21.15.57:0/1692809107 >> 172.21.15.57:0/1692809107 conn(0x5563bc418750 msgr2=0x5563bc422cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- 172.21.15.57:0/1692809107 shutdown_connections 2024-09-12T03:09:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- 172.21.15.57:0/1692809107 wait complete. 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 -- start start 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.830+0000 7fd88e7e8740 1 Processor -- start 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41c1e0 0x5563bc42a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc49faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841bc3f0 con 0x5563bc4273a0 2024-09-12T03:09:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841bd860 con 0x5563bc41c1e0 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd8841bcac0 con 0x5563bc41cb00 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd88e7e8740 10 client.0 _ll_register_callbacks cb 0x7fd86c00bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd888e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd888e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:51860/0 (socket says 172.21.15.57:51860) 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd888e00640 1 -- 172.21.15.57:0/2000155255 learned_addr learned my addr 172.21.15.57:0/2000155255 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd87be00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41c1e0 0x5563bc42a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.831+0000 7fd87be00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc49faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 -- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41c1e0 msgr2=0x5563bc42a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41c1e0 0x5563bc42a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 -- 172.21.15.57:0/2000155255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 msgr2=0x5563bc49faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc49faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:34.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 -- 172.21.15.57:0/2000155255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563bc4a00e0 con 0x5563bc41cb00 2024-09-12T03:09:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd888e00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004630 tx=0x7fd86c0115d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a47722328108bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd872a00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:09:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c015070 con 0x5563bc41cb00 2024-09-12T03:09:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fd86c034070 con 0x5563bc41cb00 2024-09-12T03:09:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.832+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c033020 con 0x5563bc41cb00 2024-09-12T03:09:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.833+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd86c03e050 con 0x5563bc41cb00 2024-09-12T03:09:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.833+0000 7fd88e7e8740 10 client.25537 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.833+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563bc4a1360 con 0x5563bc41cb00 2024-09-12T03:09:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.834+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd86c033020 con 0x5563bc41cb00 2024-09-12T03:09:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.834+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd86c004680 con 0x5563bc41cb00 2024-09-12T03:09:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 10 client.25537 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 20 client.25537 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc4a1a20 0x5563bc4a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 4 client.25537 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:09:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.836+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563bc389b80 con 0x5563bc4a1a20 2024-09-12T03:09:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.837+0000 7fd87be00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc4a1a20 0x5563bc4a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.837+0000 7fd87be00640 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc4a1a20 0x5563bc4a3e40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59089c5c2481ea2a server_cookie=6c2fdf4bc51be58a in_seq=0 out_seq=0 2024-09-12T03:09:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:34.837+0000 7fd872a00640 10 client.25537 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.107+0000 7fd872a00640 1 -- 172.21.15.57:0/2000155255 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563bc389b80 con 0x5563bc4a1a20 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.107+0000 7fd872a00640 10 client.25537 handle_command_reply: tid=0 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.107+0000 7fd88e7e8740 1 client.25537 shutdown 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 msgr2=0x5563bc42aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004630 tx=0x7fd86c0115d0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 shutdown_connections 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5563bc4a1a20 0x5563bc4a3e40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5563bc4273a0 0x5563bc49faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563bc41cb00 0x5563bc42aee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 --2- 172.21.15.57:0/2000155255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5563bc41c1e0 0x5563bc42a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 >> 172.21.15.57:0/2000155255 conn(0x5563bc418750 msgr2=0x5563bc41dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd872a00640 0 client.25537 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.108+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 shutdown_connections 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.109+0000 7fd88e7e8740 1 -- 172.21.15.57:0/2000155255 wait complete. 2024-09-12T03:09:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.110+0000 7fd88e7e8740 20 client.25537 trim_cache size 0 max 16384 2024-09-12T03:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:09:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:09:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 msgr2=0x7fd8680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 0x7fd8680a53c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c010880 comp rx=0 tx=0).stop 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 msgr2=0x7fd88410ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 0x7fd88410ae50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd88406b320 tx=0x7fd87c00b870 comp rx=0 tx=0).stop 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 shutdown_connections 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd884114ab0 0x7fd88410ae50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fd8680a2f00 0x7fd8680a53c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fd884109b10 0x7fd88410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 --2- 172.21.15.57:0/1408022646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8841091f0 0x7fd88410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.116+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 >> 172.21.15.57:0/1408022646 conn(0x7fd88406ce00 msgr2=0x7fd884070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.117+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 shutdown_connections 2024-09-12T03:09:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:35.117+0000 7fd88b800640 1 -- 172.21.15.57:0/1408022646 wait complete. 2024-09-12T03:09:35.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:34 smithi057 ceph-mon[30361]: pgmap v112: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 197 MiB/s wr, 1.90k op/s 2024-09-12T03:09:35.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:34 smithi204 ceph-mon[35013]: pgmap v112: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 197 MiB/s wr, 1.90k op/s 2024-09-12T03:09:35.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:34 smithi181 ceph-mon[35320]: pgmap v112: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 197 MiB/s wr, 1.90k op/s 2024-09-12T03:09:37.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:36 smithi057 ceph-mon[30361]: pgmap v113: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 150 MiB/s wr, 1.46k op/s 2024-09-12T03:09:37.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:36 smithi204 ceph-mon[35013]: pgmap v113: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 150 MiB/s wr, 1.46k op/s 2024-09-12T03:09:37.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:36 smithi181 ceph-mon[35320]: pgmap v113: 129 pgs: 129 active+clean; 20 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 150 MiB/s wr, 1.46k op/s 2024-09-12T03:09:39.376 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:38 smithi057 ceph-mon[30361]: pgmap v114: 129 pgs: 129 active+clean; 21 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.93k op/s 2024-09-12T03:09:39.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:38 smithi204 ceph-mon[35013]: pgmap v114: 129 pgs: 129 active+clean; 21 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.93k op/s 2024-09-12T03:09:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:38 smithi181 ceph-mon[35320]: pgmap v114: 129 pgs: 129 active+clean; 21 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 214 MiB/s wr, 1.93k op/s 2024-09-12T03:09:41.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:40 smithi057 ceph-mon[30361]: pgmap v115: 129 pgs: 129 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.71k op/s 2024-09-12T03:09:41.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:40 smithi204 ceph-mon[35013]: pgmap v115: 129 pgs: 129 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.71k op/s 2024-09-12T03:09:41.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:40 smithi181 ceph-mon[35320]: pgmap v115: 129 pgs: 129 active+clean; 21 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 209 MiB/s wr, 1.71k op/s 2024-09-12T03:09:43.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:42 smithi057 ceph-mon[30361]: pgmap v116: 129 pgs: 129 active+clean; 22 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 258 MiB/s wr, 1.92k op/s 2024-09-12T03:09:43.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:42 smithi204 ceph-mon[35013]: pgmap v116: 129 pgs: 129 active+clean; 22 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 258 MiB/s wr, 1.92k op/s 2024-09-12T03:09:43.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:42 smithi181 ceph-mon[35320]: pgmap v116: 129 pgs: 129 active+clean; 22 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 258 MiB/s wr, 1.92k op/s 2024-09-12T03:09:45.379 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:44 smithi057 ceph-mon[30361]: pgmap v117: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.56k op/s 2024-09-12T03:09:45.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:44 smithi204 ceph-mon[35013]: pgmap v117: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.56k op/s 2024-09-12T03:09:45.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:44 smithi181 ceph-mon[35320]: pgmap v117: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 230 MiB/s wr, 1.56k op/s 2024-09-12T03:09:47.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:46 smithi057 ceph-mon[30361]: pgmap v118: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.27k op/s 2024-09-12T03:09:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:46 smithi204 ceph-mon[35013]: pgmap v118: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.27k op/s 2024-09-12T03:09:47.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:46 smithi181 ceph-mon[35320]: pgmap v118: 129 pgs: 129 active+clean; 23 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 196 MiB/s wr, 1.27k op/s 2024-09-12T03:09:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 -- 172.21.15.57:0/1953151193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0102470 msgr2=0x7ff9d010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 --2- 172.21.15.57:0/1953151193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0102470 0x7ff9d010e9a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009be0 tx=0x7ff9c802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 -- 172.21.15.57:0/1953151193 shutdown_connections 2024-09-12T03:09:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 --2- 172.21.15.57:0/1953151193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0102470 0x7ff9d010e9a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 --2- 172.21.15.57:0/1953151193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0101ad0 0x7ff9d0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 --2- 172.21.15.57:0/1953151193 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0107b70 0x7ff9d0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 -- 172.21.15.57:0/1953151193 >> 172.21.15.57:0/1953151193 conn(0x7ff9d00fd7d0 msgr2=0x7ff9d00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.432+0000 7ff9d8600640 1 -- 172.21.15.57:0/1953151193 shutdown_connections 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.433+0000 7ff9d8600640 1 -- 172.21.15.57:0/1953151193 wait complete. 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 Processor -- start 2024-09-12T03:09:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 -- start start 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0101ad0 0x7ff9d010e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0107b70 0x7ff9d010cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d006e5a0 con 0x7ff9d0102470 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d006e420 con 0x7ff9d0101ad0 2024-09-12T03:09:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d006e720 con 0x7ff9d0107b70 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.434+0000 7ff9d5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0101ad0 0x7ff9d010e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:55298/0 (socket says 172.21.15.57:55298) 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 -- 172.21.15.57:0/2610400722 learned_addr learned my addr 172.21.15.57:0/2610400722 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d6600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0107b70 0x7ff9d010cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 -- 172.21.15.57:0/2610400722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0107b70 msgr2=0x7ff9d010cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0107b70 0x7ff9d010cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 -- 172.21.15.57:0/2610400722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0101ad0 msgr2=0x7ff9d010e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0101ad0 0x7ff9d010e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 -- 172.21.15.57:0/2610400722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d010d480 con 0x7ff9d0102470 2024-09-12T03:09:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.435+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400e9f0 tx=0x7ff9c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.436+0000 7ff9ce000640 1 -- 172.21.15.57:0/2610400722 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c400cd80 con 0x7ff9d0102470 2024-09-12T03:09:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.436+0000 7ff9ce000640 1 -- 172.21.15.57:0/2610400722 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4005500 con 0x7ff9d0102470 2024-09-12T03:09:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.436+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d019e310 con 0x7ff9d0102470 2024-09-12T03:09:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.436+0000 7ff9ce000640 1 -- 172.21.15.57:0/2610400722 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4010690 con 0x7ff9d0102470 2024-09-12T03:09:48.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.436+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d019e800 con 0x7ff9d0102470 2024-09-12T03:09:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.442+0000 7ff9db426740 1 Processor -- start 2024-09-12T03:09:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.442+0000 7ff9ce000640 1 -- 172.21.15.57:0/2610400722 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ff9c40046c0 con 0x7ff9d0102470 2024-09-12T03:09:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 -- start start 2024-09-12T03:09:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be81e0 0x55b006be85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006bf2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bf5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b006b4e100 con 0x55b006bf33a0 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b006af61b0 con 0x55b006be81e0 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.443+0000 7ff9db426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b006b4a0c0 con 0x55b006be8b00 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.444+0000 7ff9d5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006bf2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.444+0000 7ff9d5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006bf2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35018/0 (socket says 172.21.15.57:35018) 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.444+0000 7ff9ce000640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 0x7ff9b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.444+0000 7ff9d5c00640 1 -- 172.21.15.57:0/1669916002 learned_addr learned my addr 172.21.15.57:0/1669916002 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.445+0000 7ff9d6600640 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be81e0 0x55b006be85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.446+0000 7ff9ce000640 1 -- 172.21.15.57:0/2610400722 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff9c4099750 con 0x7ff9d0102470 2024-09-12T03:09:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.446+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bf5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d6600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 0x7ff9b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d5200640 1 -- 172.21.15.57:0/1669916002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 msgr2=0x55b006bf2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006bf2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d5200640 1 -- 172.21.15.57:0/1669916002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be81e0 msgr2=0x55b006be85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be81e0 0x55b006be85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.447+0000 7ff9d5200640 1 -- 172.21.15.57:0/1669916002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b006861210 con 0x55b006bf33a0 2024-09-12T03:09:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.448+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bf5790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40cc8e0 tx=0x7ff9c409b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ff62c0f95e6399 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.449 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.448+0000 7ff9d6600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 0x7ff9b80a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006010 tx=0x7ff9c80092c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.449+0000 7ff9bea00640 1 -- 172.21.15.57:0/1669916002 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c409b6b0 con 0x55b006bf33a0 2024-09-12T03:09:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.449+0000 7ff9bea00640 1 -- 172.21.15.57:0/1669916002 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c409fa60 con 0x55b006bf33a0 2024-09-12T03:09:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.449+0000 7ff9bea00640 1 -- 172.21.15.57:0/1669916002 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c409ad60 con 0x55b006bf33a0 2024-09-12T03:09:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.451+0000 7ff9db426740 1 -- 172.21.15.57:0/1669916002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 msgr2=0x55b006bf5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.451 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.451+0000 7ff9db426740 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bf5790 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40cc8e0 tx=0x7ff9c409b010 comp rx=0 tx=0).stop 2024-09-12T03:09:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.453+0000 7ff9db426740 1 -- 172.21.15.57:0/1669916002 shutdown_connections 2024-09-12T03:09:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.453+0000 7ff9db426740 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bf5790 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.453+0000 7ff9db426740 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006bf2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.453+0000 7ff9db426740 1 --2- 172.21.15.57:0/1669916002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be81e0 0x55b006be85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.453+0000 7ff9db426740 1 -- 172.21.15.57:0/1669916002 >> 172.21.15.57:0/1669916002 conn(0x55b006be4750 msgr2=0x55b006be4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.457+0000 7ff9db426740 1 -- 172.21.15.57:0/1669916002 shutdown_connections 2024-09-12T03:09:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.457+0000 7ff9db426740 1 -- 172.21.15.57:0/1669916002 wait complete. 2024-09-12T03:09:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.457+0000 7ff9db426740 1 -- start start 2024-09-12T03:09:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.458+0000 7ff9db426740 1 Processor -- start 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006bf0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bf0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 0x55b006bf0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55b006bf5f10 con 0x55b006be81e0 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55b006ad7090 con 0x55b006be8b00 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b006bf6090 con 0x55b006bf33a0 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.461+0000 7ff9db426740 10 client.0 _ll_register_callbacks cb 0x55b006b2bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.462+0000 7ff9d6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 0x55b006bf0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.462+0000 7ff9d6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 0x55b006bf0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35030/0 (socket says 172.21.15.57:35030) 2024-09-12T03:09:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.462+0000 7ff9d6600640 1 -- 172.21.15.57:0/2520713148 learned_addr learned my addr 172.21.15.57:0/2520713148 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.462+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bf0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 -- 172.21.15.57:0/2520713148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 msgr2=0x55b006bf0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 0x55b006bf0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 -- 172.21.15.57:0/2520713148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 msgr2=0x55b006bf0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006bf0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 -- 172.21.15.57:0/2520713148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00692d620 con 0x55b006be8b00 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9d5200640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bf0950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b006bead80 tx=0x7ff9c40ccd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56848cbdd6574844 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.463+0000 7ff9b3400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.464+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c409b6b0 con 0x55b006be8b00 2024-09-12T03:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.464+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff9c40afde0 con 0x55b006be8b00 2024-09-12T03:09:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.464+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c409fa60 con 0x55b006be8b00 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.468+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c409ab00 con 0x55b006be8b00 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.468+0000 7ff9db426740 10 client.25517 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.468+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b006bf5f10 con 0x55b006be8b00 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.468+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c409fa60 con 0x55b006be8b00 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.468+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c40bb070 con 0x55b006be8b00 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 10 client.25517 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 10 client.25517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 20 client.25517 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006bfc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 4 client.25517 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:09:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.469+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b006b5b830 con 0x55b006bf9ce0 2024-09-12T03:09:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.470+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006bfc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.471+0000 7ff9b3400640 10 client.25517 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.471+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006bfc110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a2a14df191027b server_cookie=111dd721b7b6a764 in_seq=0 out_seq=0 2024-09-12T03:09:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.472+0000 7ff9b3400640 1 -- 172.21.15.57:0/2520713148 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b006b5b830 con 0x55b006bf9ce0 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.472+0000 7ff9b3400640 10 client.25517 handle_command_reply: tid=0 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.472+0000 7ff9db426740 1 client.25517 shutdown 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.473+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 msgr2=0x55b006bf0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.473+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bf0950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b006bead80 tx=0x7ff9c40ccd90 comp rx=0 tx=0).stop 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.473+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 shutdown_connections 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.473+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006bfc110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.473+0000 7ff9b3400640 0 client.25517 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.474+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006bf33a0 0x55b006bf0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.474+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bf0950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.474+0000 7ff9db426740 1 --2- 172.21.15.57:0/2520713148 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006bf0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.474+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 >> 172.21.15.57:0/2520713148 conn(0x55b006be4750 msgr2=0x55b006bea4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.480+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 shutdown_connections 2024-09-12T03:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.480+0000 7ff9db426740 1 -- 172.21.15.57:0/2520713148 wait complete. 2024-09-12T03:09:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.481+0000 7ff9db426740 20 client.25517 trim_cache size 0 max 16384 2024-09-12T03:09:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 Processor -- start 2024-09-12T03:09:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 -- start start 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006c1dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006c1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a6f70 con 0x55b006be81e0 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a4840 con 0x55b006bf33a0 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.521+0000 7ff9db426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a26f0 con 0x55b006be8b00 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.522+0000 7ff9d6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.522+0000 7ff9d6600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:52902/0 (socket says 172.21.15.57:52902) 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.522+0000 7ff9d6600640 1 -- 172.21.15.57:0/1813734268 learned_addr learned my addr 172.21.15.57:0/1813734268 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.522+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006c1dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.522+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006c1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d6600640 1 -- 172.21.15.57:0/1813734268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 msgr2=0x55b006c1e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d6600640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006c1e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d6600640 1 -- 172.21.15.57:0/1813734268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 msgr2=0x55b006c1dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d6600640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006c1dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d6600640 1 -- 172.21.15.57:0/1813734268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b006bf13f0 con 0x55b006bf33a0 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006c1dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.523+0000 7ff9d5200640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006c1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:09:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.525+0000 7ff9d6600640 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802acb0 tx=0x7ff9c8043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54247d69febadb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.525+0000 7ff9be000640 1 -- 172.21.15.57:0/1813734268 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8036040 con 0x55b006bf33a0 2024-09-12T03:09:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.525+0000 7ff9be000640 1 -- 172.21.15.57:0/1813734268 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c804ab30 con 0x55b006bf33a0 2024-09-12T03:09:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.525+0000 7ff9be000640 1 -- 172.21.15.57:0/1813734268 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8031c70 con 0x55b006bf33a0 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 -- 172.21.15.57:0/1813734268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 msgr2=0x55b006bed670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9c802acb0 tx=0x7ff9c8043000 comp rx=0 tx=0).stop 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 -- 172.21.15.57:0/1813734268 shutdown_connections 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006bf33a0 0x55b006bed670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be8b00 0x55b006c1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 --2- 172.21.15.57:0/1813734268 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006be81e0 0x55b006c1dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.527+0000 7ff9db426740 1 -- 172.21.15.57:0/1813734268 >> 172.21.15.57:0/1813734268 conn(0x55b006be4750 msgr2=0x55b006beecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.528+0000 7ff9db426740 1 -- 172.21.15.57:0/1813734268 shutdown_connections 2024-09-12T03:09:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.528+0000 7ff9db426740 1 -- 172.21.15.57:0/1813734268 wait complete. 2024-09-12T03:09:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.529+0000 7ff9db426740 1 -- start start 2024-09-12T03:09:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.529+0000 7ff9db426740 1 Processor -- start 2024-09-12T03:09:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a2d40 con 0x55b006bf33a0 2024-09-12T03:09:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a4170 con 0x55b006be8b00 2024-09-12T03:09:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01a33d0 con 0x55b006be81e0 2024-09-12T03:09:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9d5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9d5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:35056/0 (socket says 172.21.15.57:35056) 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9d5c00640 1 -- 172.21.15.57:0/3038003913 learned_addr learned my addr 172.21.15.57:0/3038003913 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9db426740 10 client.0 _ll_register_callbacks cb 0x7ff9c001a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9d5200640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.530+0000 7ff9d6600640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 -- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 msgr2=0x55b006bed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 -- 172.21.15.57:0/3038003913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 msgr2=0x55b006bed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 -- 172.21.15.57:0/3038003913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b006bedc50 con 0x55b006be81e0 2024-09-12T03:09:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d6600640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9d5c00640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9c001a540 tx=0x7ff9c0005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298223e617fdc05c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9b3400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.531+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0032070 con 0x55b006be81e0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c804ab30 con 0x55b006be81e0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff9c002d5c0 con 0x55b006be81e0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9c003e050 con 0x55b006be81e0 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 10 client.25552 fetch_fsmap learned FSMap version 30 2024-09-12T03:09:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b006c7ead0 con 0x55b006be81e0 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c804ab30 con 0x55b006be81e0 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c0021aa0 con 0x55b006be81e0 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 10 client.25552 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 10 client.25552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 20 client.25552 populate_metadata read hostname 'smithi057' 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006c81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 4 client.25552 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.532+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b006b55b80 con 0x55b006bf9ce0 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.533+0000 7ff9d5200640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006c81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.533+0000 7ff9b3400640 10 client.25552 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.533+0000 7ff9d5200640 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006c81270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56db16deb80a1927 server_cookie=651324ef0370aa02 in_seq=0 out_seq=0 2024-09-12T03:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9b3400640 1 -- 172.21.15.57:0/3038003913 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b006b55b80 con 0x55b006bf9ce0 2024-09-12T03:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9b3400640 10 client.25552 handle_command_reply: tid=0 2024-09-12T03:09:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9db426740 1 client.25552 shutdown 2024-09-12T03:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 msgr2=0x55b006c01ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9c001a540 tx=0x7ff9c0005750 comp rx=0 tx=0).stop 2024-09-12T03:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 shutdown_connections 2024-09-12T03:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55b006bf9ce0 0x55b006c81270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.552+0000 7ff9b3400640 0 client.25552 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:09:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.553+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55b006bf33a0 0x55b006bed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.553+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b006be8b00 0x55b006bed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.553+0000 7ff9db426740 1 --2- 172.21.15.57:0/3038003913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b006be81e0 0x55b006c01ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.553+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 >> 172.21.15.57:0/3038003913 conn(0x55b006be4750 msgr2=0x55b006be9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.557+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 shutdown_connections 2024-09-12T03:09:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.557+0000 7ff9db426740 1 -- 172.21.15.57:0/3038003913 wait complete. 2024-09-12T03:09:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.558+0000 7ff9db426740 20 client.25552 trim_cache size 0 max 16384 2024-09-12T03:09:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.560+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 msgr2=0x7ff9b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.560+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 0x7ff9b80a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006010 tx=0x7ff9c80092c0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.560+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 msgr2=0x7ff9d010c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.560+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400e9f0 tx=0x7ff9c400edf0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 shutdown_connections 2024-09-12T03:09:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9d0107b70 0x7ff9d010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff9b80a2e20 0x7ff9b80a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff9d0102470 0x7ff9d010c7c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 --2- 172.21.15.57:0/2610400722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9d0101ad0 0x7ff9d010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.561+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 >> 172.21.15.57:0/2610400722 conn(0x7ff9d00fd7d0 msgr2=0x7ff9d00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.563+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 shutdown_connections 2024-09-12T03:09:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:09:48.566+0000 7ff9d8600640 1 -- 172.21.15.57:0/2610400722 wait complete. 2024-09-12T03:09:49.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:49 smithi057 ceph-mon[30361]: pgmap v119: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 1.62k op/s 2024-09-12T03:09:49.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:49 smithi204 ceph-mon[35013]: pgmap v119: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 1.62k op/s 2024-09-12T03:09:49.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:49 smithi181 ceph-mon[35320]: pgmap v119: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 1.62k op/s 2024-09-12T03:09:51.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:51 smithi057 ceph-mon[30361]: pgmap v120: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 241 MiB/s wr, 1.28k op/s 2024-09-12T03:09:51.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:51 smithi204 ceph-mon[35013]: pgmap v120: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 241 MiB/s wr, 1.28k op/s 2024-09-12T03:09:51.452 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:51 smithi181 ceph-mon[35320]: pgmap v120: 129 pgs: 129 active+clean; 24 GiB data, 49 GiB used, 1023 GiB / 1.0 TiB avail; 9.8 KiB/s rd, 241 MiB/s wr, 1.28k op/s 2024-09-12T03:09:53.374 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:53 smithi057 ceph-mon[30361]: pgmap v121: 129 pgs: 129 active+clean; 25 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 277 MiB/s wr, 1.37k op/s 2024-09-12T03:09:53.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:53 smithi204 ceph-mon[35013]: pgmap v121: 129 pgs: 129 active+clean; 25 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 277 MiB/s wr, 1.37k op/s 2024-09-12T03:09:53.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:53 smithi181 ceph-mon[35320]: pgmap v121: 129 pgs: 129 active+clean; 25 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 14 KiB/s rd, 277 MiB/s wr, 1.37k op/s 2024-09-12T03:09:55.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:55 smithi057 ceph-mon[30361]: pgmap v122: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 248 MiB/s wr, 1.13k op/s 2024-09-12T03:09:55.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:55 smithi204 ceph-mon[35013]: pgmap v122: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 248 MiB/s wr, 1.13k op/s 2024-09-12T03:09:55.454 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:55 smithi181 ceph-mon[35320]: pgmap v122: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 248 MiB/s wr, 1.13k op/s 2024-09-12T03:09:57.373 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:57 smithi057 ceph-mon[30361]: pgmap v123: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 212 MiB/s wr, 921 op/s 2024-09-12T03:09:57.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:57 smithi204 ceph-mon[35013]: pgmap v123: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 212 MiB/s wr, 921 op/s 2024-09-12T03:09:57.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:57 smithi181 ceph-mon[35320]: pgmap v123: 129 pgs: 129 active+clean; 25 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 11 KiB/s rd, 212 MiB/s wr, 921 op/s 2024-09-12T03:09:59.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:09:59 smithi057 ceph-mon[30361]: pgmap v124: 129 pgs: 129 active+clean; 25 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 290 MiB/s wr, 1.28k op/s 2024-09-12T03:09:59.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:09:59 smithi204 ceph-mon[35013]: pgmap v124: 129 pgs: 129 active+clean; 25 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 290 MiB/s wr, 1.28k op/s 2024-09-12T03:09:59.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:09:59 smithi181 ceph-mon[35320]: pgmap v124: 129 pgs: 129 active+clean; 25 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 16 KiB/s rd, 290 MiB/s wr, 1.28k op/s 2024-09-12T03:10:00.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:00 smithi057 ceph-mon[30361]: overall HEALTH_OK 2024-09-12T03:10:00.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:00 smithi204 ceph-mon[35013]: overall HEALTH_OK 2024-09-12T03:10:00.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:00 smithi181 ceph-mon[35320]: overall HEALTH_OK 2024-09-12T03:10:01.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:01 smithi204 ceph-mon[35013]: pgmap v125: 129 pgs: 129 active+clean; 26 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 1.09k op/s 2024-09-12T03:10:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:01 smithi181 ceph-mon[35320]: pgmap v125: 129 pgs: 129 active+clean; 26 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 1.09k op/s 2024-09-12T03:10:01.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:01 smithi057 ceph-mon[30361]: pgmap v125: 129 pgs: 129 active+clean; 26 GiB data, 53 GiB used, 1019 GiB / 1.0 TiB avail; 11 KiB/s rd, 245 MiB/s wr, 1.09k op/s 2024-09-12T03:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.812+0000 7ff1a5800640 1 -- 172.21.15.57:0/3909774310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a011b140 msgr2=0x7ff1a011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.812+0000 7ff1a5800640 1 --2- 172.21.15.57:0/3909774310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a011b140 0x7ff1a011d530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff188009930 tx=0x7ff18802a440 comp rx=0 tx=0).stop 2024-09-12T03:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.815+0000 7ff1a5800640 1 -- 172.21.15.57:0/3909774310 shutdown_connections 2024-09-12T03:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.816+0000 7ff1a5800640 1 --2- 172.21.15.57:0/3909774310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a011b140 0x7ff1a011d530 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.816+0000 7ff1a5800640 1 --2- 172.21.15.57:0/3909774310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a01095d0 0x7ff1a0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.816+0000 7ff1a5800640 1 --2- 172.21.15.57:0/3909774310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.816+0000 7ff1a5800640 1 -- 172.21.15.57:0/3909774310 >> 172.21.15.57:0/3909774310 conn(0x7ff1a006ce00 msgr2=0x7ff1a006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.818+0000 7ff1a5800640 1 -- 172.21.15.57:0/3909774310 shutdown_connections 2024-09-12T03:10:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.818+0000 7ff1a5800640 1 -- 172.21.15.57:0/3909774310 wait complete. 2024-09-12T03:10:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.820+0000 7ff1a5800640 1 Processor -- start 2024-09-12T03:10:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.820+0000 7ff1a5800640 1 -- start start 2024-09-12T03:10:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.820+0000 7ff1a5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.820+0000 7ff1a5800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.820+0000 7ff1a5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a011b140 0x7ff1a0111900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.821+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.821+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0118680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38918/0 (socket says 172.21.15.57:38918) 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff19e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff19e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:56806/0 (socket says 172.21.15.57:56806) 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff1a5800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0120ce0 con 0x7ff1a011b140 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff1a5800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0120b60 con 0x7ff1a01095d0 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff1a5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0120e60 con 0x7ff1a01091f0 2024-09-12T03:10:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.822+0000 7ff19ea00640 1 -- 172.21.15.57:0/322444368 learned_addr learned my addr 172.21.15.57:0/322444368 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19f400640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a011b140 0x7ff1a0111900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19e000640 1 -- 172.21.15.57:0/322444368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 msgr2=0x7ff1a0118680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19e000640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0118680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19e000640 1 -- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a011b140 msgr2=0x7ff1a0111900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19e000640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a011b140 0x7ff1a0111900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.823+0000 7ff19e000640 1 -- 172.21.15.57:0/322444368 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a01121c0 con 0x7ff1a01095d0 2024-09-12T03:10:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.825+0000 7ff19e000640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff19000b3f0 tx=0x7ff19000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.826+0000 7ff197400640 1 -- 172.21.15.57:0/322444368 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190015020 con 0x7ff1a01095d0 2024-09-12T03:10:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.826+0000 7ff197400640 1 -- 172.21.15.57:0/322444368 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff190011d10 con 0x7ff1a01095d0 2024-09-12T03:10:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.826+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a01124b0 con 0x7ff1a01095d0 2024-09-12T03:10:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.826+0000 7ff197400640 1 -- 172.21.15.57:0/322444368 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19000f2e0 con 0x7ff1a01095d0 2024-09-12T03:10:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.826+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a0116440 con 0x7ff1a01095d0 2024-09-12T03:10:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.833+0000 7ff1a87f3740 1 Processor -- start 2024-09-12T03:10:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 -- start start 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 0x55881e8b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 0x55881e8bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55881e819100 con 0x55881e8b31e0 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55881e7c11b0 con 0x55881e8be3a0 2024-09-12T03:10:01.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.834+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55881e8150c0 con 0x55881e8b3b00 2024-09-12T03:10:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.835+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 0x55881e8bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.836+0000 7ff19f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 0x55881e8b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.836+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 0x55881e8bde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38926/0 (socket says 172.21.15.57:38926) 2024-09-12T03:10:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.836+0000 7ff19ea00640 1 -- 172.21.15.57:0/946220839 learned_addr learned my addr 172.21.15.57:0/946220839 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19e000640 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19f400640 1 -- 172.21.15.57:0/946220839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 msgr2=0x55881e8bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19f400640 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 0x55881e8bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19f400640 1 -- 172.21.15.57:0/946220839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 msgr2=0x55881e8c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19f400640 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff19f400640 1 -- 172.21.15.57:0/946220839 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55881e52c210 con 0x55881e8b31e0 2024-09-12T03:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.838+0000 7ff197400640 1 -- 172.21.15.57:0/322444368 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ff190018020 con 0x7ff1a01095d0 2024-09-12T03:10:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.839+0000 7ff19e000640 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.840+0000 7ff19f400640 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 0x55881e8b35c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff18802a9b0 tx=0x7ff1880314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2841441d0c0b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.843+0000 7ff18fe00640 1 -- 172.21.15.57:0/946220839 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18803f070 con 0x55881e8b31e0 2024-09-12T03:10:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.844+0000 7ff18fe00640 1 -- 172.21.15.57:0/946220839 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff188005520 con 0x55881e8b31e0 2024-09-12T03:10:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.844+0000 7ff18fe00640 1 -- 172.21.15.57:0/946220839 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188035030 con 0x55881e8b31e0 2024-09-12T03:10:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.844+0000 7ff197400640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 0x7ff1840a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.845+0000 7ff1a87f3740 1 -- 172.21.15.57:0/946220839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 msgr2=0x55881e8b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.845+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 0x55881e8b35c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff18802a9b0 tx=0x7ff1880314b0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.845+0000 7ff197400640 1 -- 172.21.15.57:0/322444368 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff1900cee80 con 0x7ff1a01095d0 2024-09-12T03:10:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.846+0000 7ff19ea00640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 0x7ff1840a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.847+0000 7ff19ea00640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 0x7ff1840a5490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55881e8b05b0 tx=0x7ff198009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.852+0000 7ff1a87f3740 1 -- 172.21.15.57:0/946220839 shutdown_connections 2024-09-12T03:10:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.852+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.852+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b3b00 0x55881e8bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.852+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/946220839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b31e0 0x55881e8b35c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.852+0000 7ff1a87f3740 1 -- 172.21.15.57:0/946220839 >> 172.21.15.57:0/946220839 conn(0x55881e8af750 msgr2=0x55881e8afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.855+0000 7ff1a87f3740 1 -- 172.21.15.57:0/946220839 shutdown_connections 2024-09-12T03:10:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.856+0000 7ff1a87f3740 1 -- 172.21.15.57:0/946220839 wait complete. 2024-09-12T03:10:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.859+0000 7ff1a87f3740 1 -- start start 2024-09-12T03:10:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.859+0000 7ff1a87f3740 1 Processor -- start 2024-09-12T03:10:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.866+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.866+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.866+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.866+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.866+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38936/0 (socket says 172.21.15.57:38936) 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.867+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55881e8c0f70 con 0x55881e8b3b00 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.867+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55881e7a2090 con 0x55881e8be3a0 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.867+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55881e8c10f0 con 0x55881e8b31e0 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.867+0000 7ff19f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.867+0000 7ff1a87f3740 10 client.0 _ll_register_callbacks cb 0x55881e7f6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:01.869 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.869+0000 7ff19e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.869+0000 7ff19e000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38580/0 (socket says 172.21.15.57:38580) 2024-09-12T03:10:01.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.869+0000 7ff19e000640 1 -- 172.21.15.57:0/3025888255 learned_addr learned my addr 172.21.15.57:0/3025888255 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.870+0000 7ff19e000640 1 -- 172.21.15.57:0/3025888255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 msgr2=0x55881e8bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.870+0000 7ff19e000640 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.870+0000 7ff19e000640 1 -- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 msgr2=0x55881e8bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.870+0000 7ff19e000640 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.870+0000 7ff19e000640 1 -- 172.21.15.57:0/3025888255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55881e5f8620 con 0x55881e8b3b00 2024-09-12T03:10:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.874+0000 7ff19e000640 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff19000b3c0 tx=0x7ff190010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680e9f4d8fc067f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.876+0000 7ff18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:10:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.876+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19009b7a0 con 0x55881e8b3b00 2024-09-12T03:10:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.876+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff19009f0d0 con 0x55881e8b3b00 2024-09-12T03:10:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.876+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff188005520 con 0x55881e8b3b00 2024-09-12T03:10:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.877+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff19009b0a0 con 0x55881e8b3b00 2024-09-12T03:10:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.877+0000 7ff1a87f3740 10 client.15441 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.877+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55881e8ccb80 con 0x55881e8b3b00 2024-09-12T03:10:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.877+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff188005520 con 0x55881e8b3b00 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.878+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff1900b9ce0 con 0x55881e8b3b00 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 10 client.15441 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 10 client.15441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 20 client.15441 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e8cdff0 0x55881e8d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 4 client.15441 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.879+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55881e826830 con 0x55881e8cdff0 2024-09-12T03:10:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.880+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e8cdff0 0x55881e8d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.882+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e8cdff0 0x55881e8d03e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e757231c762b7b2c server_cookie=f0e7846c1bb28f9 in_seq=0 out_seq=0 2024-09-12T03:10:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.882+0000 7ff18cc00640 10 client.15441 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.884+0000 7ff18cc00640 1 -- 172.21.15.57:0/3025888255 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55881e826830 con 0x55881e8cdff0 2024-09-12T03:10:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.884+0000 7ff18cc00640 10 client.15441 handle_command_reply: tid=0 2024-09-12T03:10:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.884+0000 7ff1a87f3740 1 client.15441 shutdown 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.885+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 msgr2=0x55881e8bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.885+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff19000b3c0 tx=0x7ff190010740 comp rx=0 tx=0).stop 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 shutdown_connections 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e8cdff0 0x55881e8d03e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e8bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8bb890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3025888255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 >> 172.21.15.57:0/3025888255 conn(0x55881e8af750 msgr2=0x55881e8b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.886+0000 7ff18cc00640 0 client.15441 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.888+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 shutdown_connections 2024-09-12T03:10:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.889+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3025888255 wait complete. 2024-09-12T03:10:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.892+0000 7ff1a87f3740 20 client.15441 trim_cache size 0 max 16384 2024-09-12T03:10:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.944+0000 7ff1a87f3740 1 Processor -- start 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.946+0000 7ff1a87f3740 1 -- start start 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8b3b00 0x55881e8bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8be3a0 0x55881e8d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a01d9f40 con 0x55881e8be3a0 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a01b58c0 con 0x55881e8b3b00 2024-09-12T03:10:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.947+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0109a10 con 0x55881e8b31e0 2024-09-12T03:10:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.949+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.949+0000 7ff19e000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8b3b00 0x55881e8bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.949+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38944/0 (socket says 172.21.15.57:38944) 2024-09-12T03:10:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.949+0000 7ff19ea00640 1 -- 172.21.15.57:0/3341923959 learned_addr learned my addr 172.21.15.57:0/3341923959 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 -- 172.21.15.57:0/3341923959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8b3b00 msgr2=0x55881e8bbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8b3b00 0x55881e8bbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 -- 172.21.15.57:0/3341923959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8be3a0 msgr2=0x55881e8d5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8be3a0 0x55881e8d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 -- 172.21.15.57:0/3341923959 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55881e8ba2e0 con 0x55881e8b31e0 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.950+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff198011390 tx=0x7ff198019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d71d9bde555dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.951+0000 7ff18f400640 1 -- 172.21.15.57:0/3341923959 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19800f7a0 con 0x55881e8b31e0 2024-09-12T03:10:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.951+0000 7ff18f400640 1 -- 172.21.15.57:0/3341923959 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19800fe20 con 0x55881e8b31e0 2024-09-12T03:10:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.951+0000 7ff18f400640 1 -- 172.21.15.57:0/3341923959 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19802a680 con 0x55881e8b31e0 2024-09-12T03:10:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3341923959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 msgr2=0x55881e8aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff198011390 tx=0x7ff198019000 comp rx=0 tx=0).stop 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3341923959 shutdown_connections 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8be3a0 0x55881e8d5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8b3b00 0x55881e8bbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3341923959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8aa6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.952+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3341923959 >> 172.21.15.57:0/3341923959 conn(0x55881e8af750 msgr2=0x55881e8bd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.953+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3341923959 shutdown_connections 2024-09-12T03:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.953+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3341923959 wait complete. 2024-09-12T03:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.953+0000 7ff1a87f3740 1 -- start start 2024-09-12T03:10:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 Processor -- start 2024-09-12T03:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e937c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a01b3d80 con 0x55881e8b3b00 2024-09-12T03:10:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a01b51f0 con 0x55881e8be3a0 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a01b4450 con 0x55881e8b31e0 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.954+0000 7ff1a87f3740 10 client.0 _ll_register_callbacks cb 0x7ff190010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38950/0 (socket says 172.21.15.57:38950) 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 -- 172.21.15.57:0/3654610994 learned_addr learned my addr 172.21.15.57:0/3654610994 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:01.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19f400640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e937c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 -- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 msgr2=0x55881e937c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e937c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 -- 172.21.15.57:0/3654610994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 msgr2=0x55881e8c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.955+0000 7ff19ea00640 1 -- 172.21.15.57:0/3654610994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55881e938140 con 0x55881e8b31e0 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.956+0000 7ff19f400640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e937c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.956+0000 7ff19ea00640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff198009320 tx=0x7ff1980110f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=247b8e47c5865fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.956+0000 7ff18cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.957+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198004c40 con 0x55881e8b31e0 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.957+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7ff198004de0 con 0x55881e8b31e0 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.957+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19800fe20 con 0x55881e8b31e0 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.958+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff198043050 con 0x55881e8b31e0 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.958+0000 7ff1a87f3740 10 client.25570 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:01.958 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.958+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55881e9392f0 con 0x55881e8b31e0 2024-09-12T03:10:01.959 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.959+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff19800fe20 con 0x55881e8b31e0 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.959+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff19802a420 con 0x55881e8b31e0 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.959+0000 7ff1a87f3740 10 client.25570 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.959+0000 7ff1a87f3740 10 client.25570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.959+0000 7ff1a87f3740 20 client.25570 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.960+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e939a10 0x55881e93be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.960+0000 7ff1a87f3740 4 client.25570 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:10:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.960+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55881e8aad80 con 0x55881e939a10 2024-09-12T03:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.960+0000 7ff19e000640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e939a10 0x55881e93be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.961+0000 7ff19e000640 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e939a10 0x55881e93be30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b4fe682e82633f server_cookie=e2351d6211675e7f in_seq=0 out_seq=0 2024-09-12T03:10:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.961+0000 7ff18cc00640 10 client.25570 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff18cc00640 1 -- 172.21.15.57:0/3654610994 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55881e8aad80 con 0x55881e939a10 2024-09-12T03:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff18cc00640 10 client.25570 handle_command_reply: tid=0 2024-09-12T03:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 client.25570 shutdown 2024-09-12T03:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 msgr2=0x55881e8c2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff198009320 tx=0x7ff1980110f0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 shutdown_connections 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55881e939a10 0x55881e93be30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55881e8be3a0 0x55881e937c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55881e8b3b00 0x55881e8c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 --2- 172.21.15.57:0/3654610994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55881e8b31e0 0x55881e8c2480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff18cc00640 0 client.25570 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.975+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 >> 172.21.15.57:0/3654610994 conn(0x55881e8af750 msgr2=0x55881e8bc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.976+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 shutdown_connections 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.976+0000 7ff1a87f3740 1 -- 172.21.15.57:0/3654610994 wait complete. 2024-09-12T03:10:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.976+0000 7ff1a87f3740 20 client.25570 trim_cache size 0 max 16384 2024-09-12T03:10:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:10:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:10:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:10:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:10:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 msgr2=0x7ff1840a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 0x7ff1840a5490 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55881e8b05b0 tx=0x7ff198009500 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 msgr2=0x7ff1a0118bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff19000b3f0 tx=0x7ff19000b8b0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 shutdown_connections 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff1840a2fd0 0x7ff1840a5490 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff1a011b140 0x7ff1a0111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1a01095d0 0x7ff1a0118bc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 --2- 172.21.15.57:0/322444368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1a01091f0 0x7ff1a0118680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.978+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 >> 172.21.15.57:0/322444368 conn(0x7ff1a006ce00 msgr2=0x7ff1a011b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.979+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 shutdown_connections 2024-09-12T03:10:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:01.981+0000 7ff1a5800640 1 -- 172.21.15.57:0/322444368 wait complete. 2024-09-12T03:10:03.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:03 smithi204 ceph-mon[35013]: pgmap v126: 129 pgs: 129 active+clean; 26 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 284 MiB/s wr, 1.30k op/s 2024-09-12T03:10:03.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:03 smithi181 ceph-mon[35320]: pgmap v126: 129 pgs: 129 active+clean; 26 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 284 MiB/s wr, 1.30k op/s 2024-09-12T03:10:03.619 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:03 smithi057 ceph-mon[30361]: pgmap v126: 129 pgs: 129 active+clean; 26 GiB data, 54 GiB used, 1019 GiB / 1.0 TiB avail; 16 KiB/s rd, 284 MiB/s wr, 1.30k op/s 2024-09-12T03:10:05.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:05 smithi181 ceph-mon[35320]: pgmap v127: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 253 MiB/s wr, 1.20k op/s 2024-09-12T03:10:05.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:05 smithi057 ceph-mon[30361]: pgmap v127: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 253 MiB/s wr, 1.20k op/s 2024-09-12T03:10:05.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:05 smithi204 ceph-mon[35013]: pgmap v127: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 253 MiB/s wr, 1.20k op/s 2024-09-12T03:10:07.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:07 smithi181 ceph-mon[35320]: pgmap v128: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 216 MiB/s wr, 1.04k op/s 2024-09-12T03:10:07.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:07 smithi057 ceph-mon[30361]: pgmap v128: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 216 MiB/s wr, 1.04k op/s 2024-09-12T03:10:07.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:07 smithi204 ceph-mon[35013]: pgmap v128: 129 pgs: 129 active+clean; 26 GiB data, 55 GiB used, 1018 GiB / 1.0 TiB avail; 19 KiB/s rd, 216 MiB/s wr, 1.04k op/s 2024-09-12T03:10:08.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:08 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:08.627 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:08 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:08.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:08 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:09.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:09 smithi181 ceph-mon[35320]: pgmap v129: 129 pgs: 129 active+clean; 27 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 284 MiB/s wr, 1.46k op/s 2024-09-12T03:10:09.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:09 smithi057 ceph-mon[30361]: pgmap v129: 129 pgs: 129 active+clean; 27 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 284 MiB/s wr, 1.46k op/s 2024-09-12T03:10:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:09 smithi204 ceph-mon[35013]: pgmap v129: 129 pgs: 129 active+clean; 27 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 284 MiB/s wr, 1.46k op/s 2024-09-12T03:10:10.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:10 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:10.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:10 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:10.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:10 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:10:10.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:10 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:10.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:10 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:10.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:10 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:10:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:10 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:10 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:10.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:10 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:10:11.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:11 smithi181 ceph-mon[35320]: pgmap v130: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 21 KiB/s rd, 234 MiB/s wr, 1.27k op/s 2024-09-12T03:10:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:11 smithi057 ceph-mon[30361]: pgmap v130: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 21 KiB/s rd, 234 MiB/s wr, 1.27k op/s 2024-09-12T03:10:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:11 smithi204 ceph-mon[35013]: pgmap v130: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 21 KiB/s rd, 234 MiB/s wr, 1.27k op/s 2024-09-12T03:10:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:13 smithi181 ceph-mon[35320]: pgmap v131: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 256 MiB/s wr, 1.56k op/s 2024-09-12T03:10:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:13 smithi057 ceph-mon[30361]: pgmap v131: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 256 MiB/s wr, 1.56k op/s 2024-09-12T03:10:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:13 smithi204 ceph-mon[35013]: pgmap v131: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1017 GiB / 1.0 TiB avail; 26 KiB/s rd, 256 MiB/s wr, 1.56k op/s 2024-09-12T03:10:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 -- 172.21.15.57:0/3265645866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73701127a0 msgr2=0x7f7370114b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 --2- 172.21.15.57:0/3265645866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73701127a0 0x7f7370114b90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7368009ca0 tx=0x7f736802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 -- 172.21.15.57:0/3265645866 shutdown_connections 2024-09-12T03:10:15.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 --2- 172.21.15.57:0/3265645866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73701127a0 0x7f7370114b90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 --2- 172.21.15.57:0/3265645866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 0x7f7370112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 --2- 172.21.15.57:0/3265645866 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7370076720 0x7f7370076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.189+0000 7f7375200640 1 -- 172.21.15.57:0/3265645866 >> 172.21.15.57:0/3265645866 conn(0x7f73700ff7c0 msgr2=0x7f7370101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.190+0000 7f7375200640 1 -- 172.21.15.57:0/3265645866 shutdown_connections 2024-09-12T03:10:15.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.190+0000 7f7375200640 1 -- 172.21.15.57:0/3265645866 wait complete. 2024-09-12T03:10:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.191+0000 7f7375200640 1 Processor -- start 2024-09-12T03:10:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.192+0000 7f7375200640 1 -- start start 2024-09-12T03:10:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 0x7f73701a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73701127a0 0x7f737019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7370118350 con 0x7f73701127a0 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73701181d0 con 0x7f7370076720 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f7375200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73701184d0 con 0x7f7370077040 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.193+0000 7f736f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47110/0 (socket says 172.21.15.57:47110) 2024-09-12T03:10:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 -- 172.21.15.57:0/2572159463 learned_addr learned my addr 172.21.15.57:0/2572159463 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736ea00640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 0x7f73701a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736fe00640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73701127a0 0x7f737019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 -- 172.21.15.57:0/2572159463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 msgr2=0x7f73701a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 0x7f73701a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 -- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73701127a0 msgr2=0x7f737019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73701127a0 0x7f737019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.194+0000 7f736f400640 1 -- 172.21.15.57:0/2572159463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737019ef40 con 0x7f7370076720 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.195+0000 7f736f400640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f736000d950 tx=0x7f736000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.195+0000 7f7367e00640 1 -- 172.21.15.57:0/2572159463 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360017030 con 0x7f7370076720 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.195+0000 7f7367e00640 1 -- 172.21.15.57:0/2572159463 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360004020 con 0x7f7370076720 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.195+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737019f230 con 0x7f7370076720 2024-09-12T03:10:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.195+0000 7f7367e00640 1 -- 172.21.15.57:0/2572159463 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600047a0 con 0x7f7370076720 2024-09-12T03:10:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.196+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73701ab540 con 0x7f7370076720 2024-09-12T03:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.200+0000 7f7367e00640 1 -- 172.21.15.57:0/2572159463 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f73600041d0 con 0x7f7370076720 2024-09-12T03:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.202+0000 7f7367e00640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 0x7f73540a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.202+0000 7f7367e00640 1 -- 172.21.15.57:0/2572159463 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f73600cf1d0 con 0x7f7370076720 2024-09-12T03:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.202+0000 7f7379859740 1 Processor -- start 2024-09-12T03:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f736ea00640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 0x7f73540a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 -- start start 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a082f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560a0825a100 con 0x560a082f43b0 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560a082021b0 con 0x560a082ff570 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.203+0000 7f7379859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a082560c0 con 0x560a082f4cd0 2024-09-12T03:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47118/0 (socket says 172.21.15.57:47118) 2024-09-12T03:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 -- 172.21.15.57:0/2393985247 learned_addr learned my addr 172.21.15.57:0/2393985247 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736f400640 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a082f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 -- 172.21.15.57:0/2393985247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 msgr2=0x560a082ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 -- 172.21.15.57:0/2393985247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 msgr2=0x560a082f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a082f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.204+0000 7f736fe00640 1 -- 172.21.15.57:0/2393985247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a07f6d210 con 0x560a082ff570 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.205+0000 7f736ea00640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 0x7f73540a52b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f737019fed0 tx=0x7f735c009f20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.205+0000 7f736fe00640 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7368009ca0 tx=0x7f73680093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7752148d7e77802 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.206+0000 7f7364c00640 1 -- 172.21.15.57:0/2393985247 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368031db0 con 0x560a082ff570 2024-09-12T03:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.206+0000 7f7364c00640 1 -- 172.21.15.57:0/2393985247 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7368036030 con 0x560a082ff570 2024-09-12T03:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.206+0000 7f7364c00640 1 -- 172.21.15.57:0/2393985247 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680042a0 con 0x560a082ff570 2024-09-12T03:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 -- 172.21.15.57:0/2393985247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 msgr2=0x560a08301960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7368009ca0 tx=0x7f73680093e0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 -- 172.21.15.57:0/2393985247 shutdown_connections 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08301960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 --2- 172.21.15.57:0/2393985247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a082f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.207+0000 7f7379859740 1 -- 172.21.15.57:0/2393985247 >> 172.21.15.57:0/2393985247 conn(0x560a082f0920 msgr2=0x560a082f0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.208+0000 7f7379859740 1 -- 172.21.15.57:0/2393985247 shutdown_connections 2024-09-12T03:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.208+0000 7f7379859740 1 -- 172.21.15.57:0/2393985247 wait complete. 2024-09-12T03:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.209+0000 7f7379859740 1 -- start start 2024-09-12T03:10:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.209+0000 7f7379859740 1 Processor -- start 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 0x560a082fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560a08301fc0 con 0x560a082ff570 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560a081e3090 con 0x560a082f43b0 2024-09-12T03:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f7379859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560a08302140 con 0x560a082f4cd0 2024-09-12T03:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f736f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f736f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47122/0 (socket says 172.21.15.57:47122) 2024-09-12T03:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f736f400640 1 -- 172.21.15.57:0/3484883297 learned_addr learned my addr 172.21.15.57:0/3484883297 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.210+0000 7f736ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f7379859740 10 client.0 _ll_register_callbacks cb 0x560a08237f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 -- 172.21.15.57:0/3484883297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 msgr2=0x560a082fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 -- 172.21.15.57:0/3484883297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 msgr2=0x560a082fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736fe00640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 0x560a082fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 0x560a082fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 -- 172.21.15.57:0/3484883297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a08039620 con 0x560a082f43b0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736ea00640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736fe00640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 0x560a082fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f736f400640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7360009610 tx=0x7f736009bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7c96e31d3b7e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.211+0000 7f7359600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73600cc990 con 0x560a082f43b0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f736009f170 con 0x560a082f43b0 2024-09-12T03:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7368036030 con 0x560a082f43b0 2024-09-12T03:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73600a3f50 con 0x560a082f43b0 2024-09-12T03:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7379859740 10 client.25550 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.212+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a0830dce0 con 0x560a082f43b0 2024-09-12T03:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7368036030 con 0x560a082f43b0 2024-09-12T03:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f73600bb250 con 0x560a082f43b0 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 10 client.25550 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 20 client.25550 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0830f1a0 0x560a08311590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 4 client.25550 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a08267830 con 0x560a0830f1a0 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.213+0000 7f736ea00640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0830f1a0 0x560a08311590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.214+0000 7f736ea00640 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0830f1a0 0x560a08311590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ff299a9c0fcf16 server_cookie=eeb0662a4348cd1c in_seq=0 out_seq=0 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.214+0000 7f7359600640 10 client.25550 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.215+0000 7f7359600640 1 -- 172.21.15.57:0/3484883297 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a08267830 con 0x560a0830f1a0 2024-09-12T03:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.215+0000 7f7359600640 10 client.25550 handle_command_reply: tid=0 2024-09-12T03:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.215+0000 7f7379859740 1 client.25550 shutdown 2024-09-12T03:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 msgr2=0x560a082fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7360009610 tx=0x7f736009bb20 comp rx=0 tx=0).stop 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 shutdown_connections 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0830f1a0 0x560a08311590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082ff570 0x560a082fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a082fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 --2- 172.21.15.57:0/3484883297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082f43b0 0x560a082fc600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7359600640 0 client.25550 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 >> 172.21.15.57:0/3484883297 conn(0x560a082f0920 msgr2=0x560a082f6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 shutdown_connections 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.216+0000 7f7379859740 1 -- 172.21.15.57:0/3484883297 wait complete. 2024-09-12T03:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.217+0000 7f7379859740 20 client.25550 trim_cache size 0 max 16384 2024-09-12T03:10:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.242+0000 7f7379859740 1 Processor -- start 2024-09-12T03:10:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 -- start start 2024-09-12T03:10:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a08316e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a083173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a082f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b7e00 con 0x560a082f43b0 2024-09-12T03:10:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b56d0 con 0x560a082ff570 2024-09-12T03:10:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.243+0000 7f7379859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b3520 con 0x560a082f4cd0 2024-09-12T03:10:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.244+0000 7f736ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a083173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.244+0000 7f736f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a08316e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.244+0000 7f736f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a08316e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:38636/0 (socket says 172.21.15.57:38636) 2024-09-12T03:10:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.244+0000 7f736f400640 1 -- 172.21.15.57:0/231272906 learned_addr learned my addr 172.21.15.57:0/231272906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 -- 172.21.15.57:0/231272906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 msgr2=0x560a082f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736fe00640 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a082f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a082f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 -- 172.21.15.57:0/231272906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 msgr2=0x560a08316e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a08316e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 -- 172.21.15.57:0/231272906 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a08301cd0 con 0x560a082f4cd0 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736fe00640 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a082f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.245+0000 7f736ea00640 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a083173c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x560a082fbe70 tx=0x7f735c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bcea2a006f4a97d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.246+0000 7f735be00640 1 -- 172.21.15.57:0/231272906 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c004030 con 0x560a082f4cd0 2024-09-12T03:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.246+0000 7f735be00640 1 -- 172.21.15.57:0/231272906 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f735c0041d0 con 0x560a082f4cd0 2024-09-12T03:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.246+0000 7f735be00640 1 -- 172.21.15.57:0/231272906 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c0285c0 con 0x560a082f4cd0 2024-09-12T03:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 -- 172.21.15.57:0/231272906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 msgr2=0x560a083173c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a083173c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x560a082fbe70 tx=0x7f735c016000 comp rx=0 tx=0).stop 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 -- 172.21.15.57:0/231272906 shutdown_connections 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a082f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f4cd0 0x560a083173c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 --2- 172.21.15.57:0/231272906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f43b0 0x560a08316e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 -- 172.21.15.57:0/231272906 >> 172.21.15.57:0/231272906 conn(0x560a082f0920 msgr2=0x560a082faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 -- 172.21.15.57:0/231272906 shutdown_connections 2024-09-12T03:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.247+0000 7f7379859740 1 -- 172.21.15.57:0/231272906 wait complete. 2024-09-12T03:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.248+0000 7f7379859740 1 -- start start 2024-09-12T03:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.248+0000 7f7379859740 1 Processor -- start 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 0x560a083076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 0x560a08307c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b3b90 con 0x560a082f4cd0 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b5000 con 0x560a082ff570 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f73701b4260 con 0x560a082f43b0 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f7379859740 10 client.0 _ll_register_callbacks cb 0x7f736009b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f736fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f736fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:47130/0 (socket says 172.21.15.57:47130) 2024-09-12T03:10:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.249+0000 7f736fe00640 1 -- 172.21.15.57:0/1879230789 learned_addr learned my addr 172.21.15.57:0/1879230789 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.250+0000 7f736f400640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 0x560a083076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736ea00640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 0x560a08307c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736fe00640 1 -- 172.21.15.57:0/1879230789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 msgr2=0x560a083076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736fe00640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 0x560a083076d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736fe00640 1 -- 172.21.15.57:0/1879230789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 msgr2=0x560a08307c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736fe00640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 0x560a08307c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736fe00640 1 -- 172.21.15.57:0/1879230789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a083032f0 con 0x560a082ff570 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.251+0000 7f736f400640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 0x560a083076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.252+0000 7f736ea00640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 0x560a08307c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.252+0000 7f736fe00640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x560a082f6f50 tx=0x7f7368005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=546075513febfb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.252+0000 7f7359600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.252+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368051070 con 0x560a082ff570 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.253+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f7368031660 con 0x560a082ff570 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.253+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f735c0041d0 con 0x560a082ff570 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.253+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f736805b050 con 0x560a082ff570 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.253+0000 7f7379859740 10 client.25553 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.253+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a0838a9e0 con 0x560a082ff570 2024-09-12T03:10:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f735c0041d0 con 0x560a082ff570 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7368043070 con 0x560a082ff570 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 10 client.25553 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 20 client.25553 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0838b140 0x560a0838d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 4 client.25553 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.254+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a08261b80 con 0x560a0838b140 2024-09-12T03:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.255+0000 7f736f400640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0838b140 0x560a0838d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.256+0000 7f736f400640 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0838b140 0x560a0838d560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c1d09399c42737 server_cookie=e87dd765199cb2cf in_seq=0 out_seq=0 2024-09-12T03:10:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.256+0000 7f7359600640 10 client.25553 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7359600640 1 -- 172.21.15.57:0/1879230789 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a08261b80 con 0x560a0838b140 2024-09-12T03:10:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7359600640 10 client.25553 handle_command_reply: tid=0 2024-09-12T03:10:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7379859740 1 client.25553 shutdown 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 msgr2=0x560a08302ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x560a082f6f50 tx=0x7f7368005df0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.268+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 shutdown_connections 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560a0838b140 0x560a0838d560 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560a082ff570 0x560a08302ac0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560a082f4cd0 0x560a08307c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 --2- 172.21.15.57:0/1879230789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560a082f43b0 0x560a083076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 >> 172.21.15.57:0/1879230789 conn(0x560a082f0920 msgr2=0x560a082f5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7359600640 0 client.25553 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 shutdown_connections 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 1 -- 172.21.15.57:0/1879230789 wait complete. 2024-09-12T03:10:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.269+0000 7f7379859740 20 client.25553 trim_cache size 0 max 16384 2024-09-12T03:10:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:10:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:10:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:10:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 msgr2=0x7f73540a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 0x7f73540a52b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f737019fed0 tx=0x7f735c009f20 comp rx=0 tx=0).stop 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 msgr2=0x7f73701a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f736000d950 tx=0x7f736000de10 comp rx=0 tx=0).stop 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 shutdown_connections 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f73540a2df0 0x7f73540a52b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f73701127a0 0x7f737019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7370077040 0x7f73701a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 --2- 172.21.15.57:0/2572159463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7370076720 0x7f73701a45c0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f736000d950 tx=0x7f736000de10 comp rx=0 tx=0).stop 2024-09-12T03:10:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 >> 172.21.15.57:0/2572159463 conn(0x7f73700ff7c0 msgr2=0x7f7370101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 shutdown_connections 2024-09-12T03:10:15.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:15.271+0000 7f7375200640 1 -- 172.21.15.57:0/2572159463 wait complete. 2024-09-12T03:10:15.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:15 smithi181 ceph-mon[35320]: pgmap v132: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 22 KiB/s rd, 207 MiB/s wr, 1.42k op/s 2024-09-12T03:10:15.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:15 smithi057 ceph-mon[30361]: pgmap v132: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 22 KiB/s rd, 207 MiB/s wr, 1.42k op/s 2024-09-12T03:10:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:15 smithi204 ceph-mon[35013]: pgmap v132: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 22 KiB/s rd, 207 MiB/s wr, 1.42k op/s 2024-09-12T03:10:17.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:17 smithi057 ceph-mon[30361]: pgmap v133: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 172 MiB/s wr, 1.23k op/s 2024-09-12T03:10:17.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:17 smithi204 ceph-mon[35013]: pgmap v133: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 172 MiB/s wr, 1.23k op/s 2024-09-12T03:10:17.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:17 smithi181 ceph-mon[35320]: pgmap v133: 129 pgs: 129 active+clean; 27 GiB data, 56 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 172 MiB/s wr, 1.23k op/s 2024-09-12T03:10:19.627 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:19 smithi057 ceph-mon[30361]: pgmap v134: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.59k op/s 2024-09-12T03:10:19.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:19 smithi204 ceph-mon[35013]: pgmap v134: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.59k op/s 2024-09-12T03:10:19.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:19 smithi181 ceph-mon[35320]: pgmap v134: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.59k op/s 2024-09-12T03:10:21.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:21 smithi057 ceph-mon[30361]: pgmap v135: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 143 MiB/s wr, 1.42k op/s 2024-09-12T03:10:21.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:21 smithi204 ceph-mon[35013]: pgmap v135: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 143 MiB/s wr, 1.42k op/s 2024-09-12T03:10:21.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:21 smithi181 ceph-mon[35320]: pgmap v135: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 12 KiB/s rd, 143 MiB/s wr, 1.42k op/s 2024-09-12T03:10:23.622 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:23 smithi057 ceph-mon[30361]: pgmap v136: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 195 MiB/s wr, 1.96k op/s 2024-09-12T03:10:23.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:23 smithi204 ceph-mon[35013]: pgmap v136: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 195 MiB/s wr, 1.96k op/s 2024-09-12T03:10:23.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:23 smithi181 ceph-mon[35320]: pgmap v136: 129 pgs: 129 active+clean; 27 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 13 KiB/s rd, 195 MiB/s wr, 1.96k op/s 2024-09-12T03:10:25.621 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:25 smithi057 ceph-mon[30361]: pgmap v137: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 172 MiB/s wr, 1.79k op/s 2024-09-12T03:10:25.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:25 smithi204 ceph-mon[35013]: pgmap v137: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 172 MiB/s wr, 1.79k op/s 2024-09-12T03:10:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:25 smithi181 ceph-mon[35320]: pgmap v137: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 172 MiB/s wr, 1.79k op/s 2024-09-12T03:10:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:27 smithi181 ceph-mon[35320]: pgmap v138: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 166 MiB/s wr, 1.73k op/s 2024-09-12T03:10:27.874 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:27 smithi057 ceph-mon[30361]: pgmap v138: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 166 MiB/s wr, 1.73k op/s 2024-09-12T03:10:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:27 smithi204 ceph-mon[35013]: pgmap v138: 129 pgs: 129 active+clean; 27 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 166 MiB/s wr, 1.73k op/s 2024-09-12T03:10:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 -- 172.21.15.57:0/2702960958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a810e0f0 msgr2=0x7f04a8110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 --2- 172.21.15.57:0/2702960958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a810e0f0 0x7f04a8110540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009900 tx=0x7f04a002a480 comp rx=0 tx=0).stop 2024-09-12T03:10:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 -- 172.21.15.57:0/2702960958 shutdown_connections 2024-09-12T03:10:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 --2- 172.21.15.57:0/2702960958 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a810e0f0 0x7f04a8110540 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 --2- 172.21.15.57:0/2702960958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a8074f40 0x7f04a810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 --2- 172.21.15.57:0/2702960958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074620 0x7f04a8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 -- 172.21.15.57:0/2702960958 >> 172.21.15.57:0/2702960958 conn(0x7f04a80fd5d0 msgr2=0x7f04a80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 -- 172.21.15.57:0/2702960958 shutdown_connections 2024-09-12T03:10:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.487+0000 7f04a7e00640 1 -- 172.21.15.57:0/2702960958 wait complete. 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 Processor -- start 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 -- start start 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 0x7f04a819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a810e0f0 0x7f04a819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8113ef0 con 0x7f04a8074620 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8113d70 con 0x7f04a8074f40 2024-09-12T03:10:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8114070 con 0x7f04a810e0f0 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 0x7f04a819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 0x7f04a819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:59352/0 (socket says 172.21.15.57:59352) 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.489+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51490/0 (socket says 172.21.15.57:51490) 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6000640 1 -- 172.21.15.57:0/947907002 learned_addr learned my addr 172.21.15.57:0/947907002 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a7400640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a810e0f0 0x7f04a819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6a00640 1 -- 172.21.15.57:0/947907002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a810e0f0 msgr2=0x7f04a819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6a00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a810e0f0 0x7f04a819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6a00640 1 -- 172.21.15.57:0/947907002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 msgr2=0x7f04a819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6a00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 0x7f04a819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.490+0000 7f04a6a00640 1 -- 172.21.15.57:0/947907002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a819f3f0 con 0x7f04a8074620 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f04a6a00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f049800eb00 tx=0x7f049800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f049f400640 1 -- 172.21.15.57:0/947907002 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498010430 con 0x7f04a8074620 2024-09-12T03:10:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f049f400640 1 -- 172.21.15.57:0/947907002 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0498010ab0 con 0x7f04a8074620 2024-09-12T03:10:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f049f400640 1 -- 172.21.15.57:0/947907002 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498017bb0 con 0x7f04a8074620 2024-09-12T03:10:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a819f6e0 con 0x7f04a8074620 2024-09-12T03:10:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.491+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a81a6f70 con 0x7f04a8074620 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.496+0000 7f04b0a58740 1 Processor -- start 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.496+0000 7f049f400640 1 -- 172.21.15.57:0/947907002 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f0498010c60 con 0x7f04a8074620 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.498+0000 7f04b0a58740 1 -- start start 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.499+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.499+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73b70 0x55e6e3d8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.499+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.500+0000 7f04b0a58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3cd9100 con 0x55e6e3d8f580 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.500+0000 7f04b0a58740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3c811b0 con 0x55e6e3d73b70 2024-09-12T03:10:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.500+0000 7f04b0a58740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3cd50c0 con 0x55e6e3d73250 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.500+0000 7f049f400640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 0x7f048c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.501+0000 7f049f400640 1 -- 172.21.15.57:0/947907002 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f04980d0140 con 0x7f04a8074620 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:51502/0 (socket says 172.21.15.57:51502) 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 -- 172.21.15.57:0/2099257310 learned_addr learned my addr 172.21.15.57:0/2099257310 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 -- 172.21.15.57:0/2099257310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 msgr2=0x55e6e3d73630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d73630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 -- 172.21.15.57:0/2099257310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73b70 msgr2=0x55e6e3d8f040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73b70 0x55e6e3d8f040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.502+0000 7f04a6000640 1 -- 172.21.15.57:0/2099257310 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e39ec210 con 0x55e6e3d8f580 2024-09-12T03:10:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.503+0000 7f04a6000640 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f049400ac20 tx=0x7f049400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4008a7def099a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.515+0000 7f04a7400640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 0x7f048c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.515+0000 7f0493e00640 1 -- 172.21.15.57:0/2099257310 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049401d070 con 0x55e6e3d8f580 2024-09-12T03:10:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.515+0000 7f0493e00640 1 -- 172.21.15.57:0/2099257310 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494018b20 con 0x55e6e3d8f580 2024-09-12T03:10:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.515+0000 7f0493e00640 1 -- 172.21.15.57:0/2099257310 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494010b80 con 0x55e6e3d8f580 2024-09-12T03:10:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.517+0000 7f04b0a58740 1 -- 172.21.15.57:0/2099257310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 msgr2=0x55e6e3d91970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.517+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f049400ac20 tx=0x7f049400f790 comp rx=0 tx=0).stop 2024-09-12T03:10:28.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.521+0000 7f04a7400640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 0x7f048c0a52b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a0031550 tx=0x7f04a00046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.522+0000 7f04b0a58740 1 -- 172.21.15.57:0/2099257310 shutdown_connections 2024-09-12T03:10:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.522+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d91970 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.522+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73b70 0x55e6e3d8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.522+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2099257310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.523+0000 7f04b0a58740 1 -- 172.21.15.57:0/2099257310 >> 172.21.15.57:0/2099257310 conn(0x55e6e3d6f920 msgr2=0x55e6e3d6fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.532+0000 7f04b0a58740 1 -- 172.21.15.57:0/2099257310 shutdown_connections 2024-09-12T03:10:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.532+0000 7f04b0a58740 1 -- 172.21.15.57:0/2099257310 wait complete. 2024-09-12T03:10:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.535+0000 7f04b0a58740 1 -- start start 2024-09-12T03:10:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.538+0000 7f04b0a58740 1 Processor -- start 2024-09-12T03:10:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.539+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 0x55e6e3d80980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.539+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 0x55e6e3d80ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.539+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d7bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.540+0000 7f04b0a58740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3d76210 con 0x55e6e3d8f580 2024-09-12T03:10:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.541+0000 7f04b0a58740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3c62090 con 0x55e6e3d73250 2024-09-12T03:10:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.541+0000 7f04b0a58740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e6e3d76390 con 0x55e6e3d73b70 2024-09-12T03:10:28.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.541+0000 7f04b0a58740 10 client.0 _ll_register_callbacks cb 0x55e6e3cb6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.541+0000 7f04a6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 0x55e6e3d80ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.541+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 0x55e6e3d80980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.542+0000 7f04a6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 0x55e6e3d80ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:54052/0 (socket says 172.21.15.57:54052) 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.542+0000 7f04a6000640 1 -- 172.21.15.57:0/1249015380 learned_addr learned my addr 172.21.15.57:0/1249015380 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.543+0000 7f04a6a00640 1 -- 172.21.15.57:0/1249015380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 msgr2=0x55e6e3d80ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.543+0000 7f04a6a00640 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 0x55e6e3d80ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.543+0000 7f04a6a00640 1 -- 172.21.15.57:0/1249015380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 msgr2=0x55e6e3d7bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.543+0000 7f04a6a00640 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d7bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.543+0000 7f04a6a00640 1 -- 172.21.15.57:0/1249015380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e3ab8620 con 0x55e6e3d73250 2024-09-12T03:10:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.547+0000 7f04a6a00640 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 0x55e6e3d80980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04980cd480 tx=0x7f049809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5304d49f7b870d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.547+0000 7f0490c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.548+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980b4020 con 0x55e6e3d73250 2024-09-12T03:10:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.548+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f04980988c0 con 0x55e6e3d73250 2024-09-12T03:10:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.548+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0494018b20 con 0x55e6e3d73250 2024-09-12T03:10:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.549+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04980ba050 con 0x55e6e3d73250 2024-09-12T03:10:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.549+0000 7f04b0a58740 10 client.25562 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.549+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6e3d8a640 con 0x55e6e3d73250 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.557+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494018b20 con 0x55e6e3d73250 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.557+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f04980a0280 con 0x55e6e3d73250 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 10 client.25562 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 20 client.25562 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3d8bb00 0x55e6e3d8df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 4 client.25562 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.560+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6e3ce6830 con 0x55e6e3d8bb00 2024-09-12T03:10:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.561+0000 7f04a6000640 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3d8bb00 0x55e6e3d8df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.567+0000 7f0490c00640 10 client.25562 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.567+0000 7f04a6000640 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3d8bb00 0x55e6e3d8df30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658f44a5ab8c38d server_cookie=86242d7ec9bf3734 in_seq=0 out_seq=0 2024-09-12T03:10:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.570+0000 7f0490c00640 1 -- 172.21.15.57:0/1249015380 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6e3ce6830 con 0x55e6e3d8bb00 2024-09-12T03:10:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.570+0000 7f0490c00640 10 client.25562 handle_command_reply: tid=0 2024-09-12T03:10:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.570+0000 7f04b0a58740 1 client.25562 shutdown 2024-09-12T03:10:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 msgr2=0x55e6e3d80980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 0x55e6e3d80980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f04980cd480 tx=0x7f049809be80 comp rx=0 tx=0).stop 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 shutdown_connections 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3d8bb00 0x55e6e3d8df30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d8f580 0x55e6e3d7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73b70 0x55e6e3d80ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 --2- 172.21.15.57:0/1249015380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d73250 0x55e6e3d80980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 >> 172.21.15.57:0/1249015380 conn(0x55e6e3d6f920 msgr2=0x55e6e3d75680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.573+0000 7f0490c00640 0 client.25562 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.574+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 shutdown_connections 2024-09-12T03:10:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.574+0000 7f04b0a58740 1 -- 172.21.15.57:0/1249015380 wait complete. 2024-09-12T03:10:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.574+0000 7f04b0a58740 20 client.25562 trim_cache size 0 max 16384 2024-09-12T03:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.642+0000 7f04b0a58740 1 Processor -- start 2024-09-12T03:10:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.647+0000 7f04b0a58740 1 -- start start 2024-09-12T03:10:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.647+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d96fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:54054/0 (socket says 172.21.15.57:54054) 2024-09-12T03:10:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04a6a00640 1 -- 172.21.15.57:0/2261251650 learned_addr learned my addr 172.21.15.57:0/2261251650 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3d974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f04a81ab1c0 con 0x55e6e3d73b70 2024-09-12T03:10:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a81a8a90 con 0x55e6e3d8f580 2024-09-12T03:10:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.649+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04a8073b60 con 0x55e6e3d73250 2024-09-12T03:10:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.650+0000 7f04a7400640 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3d974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.650+0000 7f04a6a00640 1 -- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 msgr2=0x55e6e3d974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.651+0000 7f04a6a00640 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3d974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.651+0000 7f04a6a00640 1 -- 172.21.15.57:0/2261251650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 msgr2=0x55e6e3d96fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.651+0000 7f04a6a00640 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d96fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.651+0000 7f04a6a00640 1 -- 172.21.15.57:0/2261251650 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e3d91970 con 0x55e6e3d73250 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.651+0000 7f04a7400640 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3d974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.652+0000 7f04a6a00640 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f049809a500 tx=0x7f049809c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374336865e55525d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.653+0000 7f0493400640 1 -- 172.21.15.57:0/2261251650 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980b4070 con 0x55e6e3d73250 2024-09-12T03:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.653+0000 7f0493400640 1 -- 172.21.15.57:0/2261251650 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04980a0600 con 0x55e6e3d73250 2024-09-12T03:10:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.653+0000 7f0493400640 1 -- 172.21.15.57:0/2261251650 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980cd700 con 0x55e6e3d73250 2024-09-12T03:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.658+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 msgr2=0x55e6e3d8eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.658+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f049809a500 tx=0x7f049809c3d0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.660+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 shutdown_connections 2024-09-12T03:10:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.660+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3d974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.660+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d96fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.660+0000 7f04b0a58740 1 --2- 172.21.15.57:0/2261251650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d8eee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.660+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 >> 172.21.15.57:0/2261251650 conn(0x55e6e3d6f920 msgr2=0x55e6e3d8e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.661+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 shutdown_connections 2024-09-12T03:10:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.661+0000 7f04b0a58740 1 -- 172.21.15.57:0/2261251650 wait complete. 2024-09-12T03:10:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.661+0000 7f04b0a58740 1 -- start start 2024-09-12T03:10:28.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.662+0000 7f04b0a58740 1 Processor -- start 2024-09-12T03:10:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.664+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.665+0000 7f04b0a58740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.665+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.665+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d84d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:54070/0 (socket says 172.21.15.57:54070) 2024-09-12T03:10:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.665+0000 7f04a6a00640 1 -- 172.21.15.57:0/538249526 learned_addr learned my addr 172.21.15.57:0/538249526 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.666+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3e00780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.666+0000 7f04a6000640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.666+0000 7f04a7400640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3e00780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.667+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f04a80741d0 con 0x55e6e3d73b70 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.667+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f04a81a83c0 con 0x55e6e3d8f580 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.667+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f04a81a7620 con 0x55e6e3d73250 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.667+0000 7f04a7400640 1 -- 172.21.15.57:0/538249526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 msgr2=0x55e6e3d84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.667+0000 7f04a7400640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d84d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.668+0000 7f04a7400640 1 -- 172.21.15.57:0/538249526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 msgr2=0x55e6e3d852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.668+0000 7f04a7400640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.668+0000 7f04a7400640 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e3d85b40 con 0x55e6e3d8f580 2024-09-12T03:10:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.669+0000 7f04b0a58740 10 client.0 _ll_register_callbacks cb 0x7f049809c090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.669+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04980a0600 con 0x55e6e3d8f580 2024-09-12T03:10:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.669+0000 7f04a7400640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3e00780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04a002a9a0 tx=0x7f04a00041e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2464191fbe5d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.669+0000 7f0490c00640 10 client.25568 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.671+0000 7f0490c00640 1 -- 172.21.15.57:0/538249526 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0058050 con 0x55e6e3d8f580 2024-09-12T03:10:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.671+0000 7f0490c00640 1 -- 172.21.15.57:0/538249526 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f04a0003d50 con 0x55e6e3d8f580 2024-09-12T03:10:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.671+0000 7f0490c00640 1 -- 172.21.15.57:0/538249526 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04a0058050 con 0x55e6e3d8f580 2024-09-12T03:10:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.672+0000 7f04b0a58740 10 client.25568 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.672+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e6e3e01a70 con 0x55e6e3d8f580 2024-09-12T03:10:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.673+0000 7f0490c00640 1 -- 172.21.15.57:0/538249526 <== mon.2 v2:172.21.15.181:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f04a00315b0 con 0x55e6e3d8f580 2024-09-12T03:10:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.673+0000 7f04b0a58740 10 client.25568 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.673+0000 7f04b0a58740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.673+0000 7f04b0a58740 20 client.25568 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.674+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3e02200 0x55e6e3e046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.674+0000 7f04b0a58740 4 client.25568 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:10:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.674+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6e3ce0b80 con 0x55e6e3e02200 2024-09-12T03:10:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.675+0000 7f04a6a00640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3e02200 0x55e6e3e046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.676+0000 7f04a6a00640 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3e02200 0x55e6e3e046c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce38a65e9985d6d6 server_cookie=40405c5881dd4396 in_seq=0 out_seq=0 2024-09-12T03:10:28.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.676+0000 7f0490c00640 10 client.25568 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.857+0000 7f0490c00640 1 -- 172.21.15.57:0/538249526 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6e3ce0b80 con 0x55e6e3e02200 2024-09-12T03:10:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.858+0000 7f0490c00640 10 client.25568 handle_command_reply: tid=0 2024-09-12T03:10:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.858+0000 7f04b0a58740 1 client.25568 shutdown 2024-09-12T03:10:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.859+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 msgr2=0x55e6e3e00780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.859+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3e00780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f04a002a9a0 tx=0x7f04a00041e0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.859+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 shutdown_connections 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.859+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e6e3e02200 0x55e6e3e046c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.859+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e6e3d8f580 0x55e6e3e00780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.860+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e6e3d73b70 0x55e6e3d852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.860+0000 7f0490c00640 0 client.25568 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.860+0000 7f04b0a58740 1 --2- 172.21.15.57:0/538249526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6e3d73250 0x55e6e3d84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.860+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 >> 172.21.15.57:0/538249526 conn(0x55e6e3d6f920 msgr2=0x55e6e3d6e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.861+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 shutdown_connections 2024-09-12T03:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.861+0000 7f04b0a58740 1 -- 172.21.15.57:0/538249526 wait complete. 2024-09-12T03:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.861+0000 7f04b0a58740 20 client.25568 trim_cache size 0 max 16384 2024-09-12T03:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.864+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 msgr2=0x7f048c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.864+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 0x7f048c0a52b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04a0031550 tx=0x7f04a00046c0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.864+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 msgr2=0x7f04a819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.864+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f049800eb00 tx=0x7f049800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 shutdown_connections 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04a810e0f0 0x7f04a819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a8074f40 0x7f04a819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f048c0a2df0 0x7f048c0a52b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 --2- 172.21.15.57:0/947907002 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f04a8074620 0x7f04a819a400 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 >> 172.21.15.57:0/947907002 conn(0x7f04a80fd5d0 msgr2=0x7f04a80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.865+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 shutdown_connections 2024-09-12T03:10:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:28.866+0000 7f04a7e00640 1 -- 172.21.15.57:0/947907002 wait complete. 2024-09-12T03:10:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:29 smithi181 ceph-mon[35320]: pgmap v139: 129 pgs: 129 active+clean; 28 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 2.23k op/s 2024-09-12T03:10:29.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:29 smithi057 ceph-mon[30361]: pgmap v139: 129 pgs: 129 active+clean; 28 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 2.23k op/s 2024-09-12T03:10:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:29 smithi204 ceph-mon[35013]: pgmap v139: 129 pgs: 129 active+clean; 28 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 13 KiB/s rd, 210 MiB/s wr, 2.23k op/s 2024-09-12T03:10:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:31 smithi181 ceph-mon[35320]: pgmap v140: 129 pgs: 129 active+clean; 28 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 210 MiB/s wr, 2.09k op/s 2024-09-12T03:10:31.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:31 smithi057 ceph-mon[30361]: pgmap v140: 129 pgs: 129 active+clean; 28 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 210 MiB/s wr, 2.09k op/s 2024-09-12T03:10:31.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:31 smithi204 ceph-mon[35013]: pgmap v140: 129 pgs: 129 active+clean; 28 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 8.8 KiB/s rd, 210 MiB/s wr, 2.09k op/s 2024-09-12T03:10:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:33 smithi181 ceph-mon[35320]: pgmap v141: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 11 KiB/s rd, 247 MiB/s wr, 2.38k op/s 2024-09-12T03:10:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:33 smithi204 ceph-mon[35013]: pgmap v141: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 11 KiB/s rd, 247 MiB/s wr, 2.38k op/s 2024-09-12T03:10:34.680 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:33 smithi057 ceph-mon[30361]: pgmap v141: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 11 KiB/s rd, 247 MiB/s wr, 2.38k op/s 2024-09-12T03:10:35.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:35 smithi057 ceph-mon[30361]: pgmap v142: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 175 MiB/s wr, 1.74k op/s 2024-09-12T03:10:35.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:35 smithi204 ceph-mon[35013]: pgmap v142: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 175 MiB/s wr, 1.74k op/s 2024-09-12T03:10:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:35 smithi181 ceph-mon[35320]: pgmap v142: 129 pgs: 129 active+clean; 29 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 175 MiB/s wr, 1.74k op/s 2024-09-12T03:10:37.884 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:37 smithi057 ceph-mon[30361]: pgmap v143: 129 pgs: 129 active+clean; 29 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 169 MiB/s wr, 1.68k op/s 2024-09-12T03:10:37.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:37 smithi204 ceph-mon[35013]: pgmap v143: 129 pgs: 129 active+clean; 29 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 169 MiB/s wr, 1.68k op/s 2024-09-12T03:10:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:37 smithi181 ceph-mon[35320]: pgmap v143: 129 pgs: 129 active+clean; 29 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 169 MiB/s wr, 1.68k op/s 2024-09-12T03:10:39.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:39 smithi204 ceph-mon[35013]: pgmap v144: 129 pgs: 129 active+clean; 30 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 2.05k op/s 2024-09-12T03:10:39.896 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:39 smithi057 ceph-mon[30361]: pgmap v144: 129 pgs: 129 active+clean; 30 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 2.05k op/s 2024-09-12T03:10:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:39 smithi181 ceph-mon[35320]: pgmap v144: 129 pgs: 129 active+clean; 30 GiB data, 62 GiB used, 1010 GiB / 1.0 TiB avail; 20 KiB/s rd, 222 MiB/s wr, 2.05k op/s 2024-09-12T03:10:41.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:41 smithi057 ceph-mon[30361]: pgmap v145: 129 pgs: 129 active+clean; 30 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 15 KiB/s rd, 207 MiB/s wr, 1.75k op/s 2024-09-12T03:10:41.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:41 smithi204 ceph-mon[35013]: pgmap v145: 129 pgs: 129 active+clean; 30 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 15 KiB/s rd, 207 MiB/s wr, 1.75k op/s 2024-09-12T03:10:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:41 smithi181 ceph-mon[35320]: pgmap v145: 129 pgs: 129 active+clean; 30 GiB data, 63 GiB used, 1010 GiB / 1.0 TiB avail; 15 KiB/s rd, 207 MiB/s wr, 1.75k op/s 2024-09-12T03:10:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 -- 172.21.15.57:0/3275573972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b410e0f0 msgr2=0x7f02b4110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 --2- 172.21.15.57:0/3275573972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b410e0f0 0x7f02b4110540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009ca0 tx=0x7f02ac02a950 comp rx=0 tx=0).stop 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 -- 172.21.15.57:0/3275573972 shutdown_connections 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 --2- 172.21.15.57:0/3275573972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b410e0f0 0x7f02b4110540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 --2- 172.21.15.57:0/3275573972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074f40 0x7f02b410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 --2- 172.21.15.57:0/3275573972 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02b4074620 0x7f02b4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 -- 172.21.15.57:0/3275573972 >> 172.21.15.57:0/3275573972 conn(0x7f02b40fd5d0 msgr2=0x7f02b40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.606+0000 7f02b9000640 1 -- 172.21.15.57:0/3275573972 shutdown_connections 2024-09-12T03:10:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.607+0000 7f02b9000640 1 -- 172.21.15.57:0/3275573972 wait complete. 2024-09-12T03:10:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.617+0000 7f02b9000640 1 Processor -- start 2024-09-12T03:10:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.617+0000 7f02b9000640 1 -- start start 2024-09-12T03:10:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 0x7f02b410dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 0x7f02b410ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02b410e0f0 0x7f02b410b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4113ef0 con 0x7f02b410e0f0 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4113d70 con 0x7f02b4074620 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.618+0000 7f02b9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02b4114070 con 0x7f02b4074f40 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.619+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 0x7f02b410dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.619+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 0x7f02b410dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38466/0 (socket says 172.21.15.57:38466) 2024-09-12T03:10:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.619+0000 7f02b3400640 1 -- 172.21.15.57:0/3454097071 learned_addr learned my addr 172.21.15.57:0/3454097071 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 0x7f02b410ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 -- 172.21.15.57:0/3454097071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 msgr2=0x7f02b410dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 0x7f02b410dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 -- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02b410e0f0 msgr2=0x7f02b410b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02b410e0f0 0x7f02b410b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 -- 172.21.15.57:0/3454097071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b4199380 con 0x7f02b4074f40 2024-09-12T03:10:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.620+0000 7f02abe00640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 0x7f02b410ab80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f029c005fd0 tx=0x7f029c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.621+0000 7f02b0c00640 1 -- 172.21.15.57:0/3454097071 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c009c30 con 0x7f02b4074f40 2024-09-12T03:10:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.621+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b4199670 con 0x7f02b4074f40 2024-09-12T03:10:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.622+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b4199c10 con 0x7f02b4074f40 2024-09-12T03:10:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.622+0000 7f02b0c00640 1 -- 172.21.15.57:0/3454097071 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f029c009dd0 con 0x7f02b4074f40 2024-09-12T03:10:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.622+0000 7f02b0c00640 1 -- 172.21.15.57:0/3454097071 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c010460 con 0x7f02b4074f40 2024-09-12T03:10:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.630+0000 7f02b0c00640 1 -- 172.21.15.57:0/3454097071 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f029c010600 con 0x7f02b4074f40 2024-09-12T03:10:42.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.639+0000 7f02b0c00640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 0x7f02980a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.641+0000 7f02b0c00640 1 -- 172.21.15.57:0/3454097071 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f029c0d0300 con 0x7f02b4074f40 2024-09-12T03:10:42.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.644+0000 7f02b3400640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 0x7f02980a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.682+0000 7f02b3400640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 0x7f02980a52b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009c20 tx=0x7f02a4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.686+0000 7f02bd5c4740 1 Processor -- start 2024-09-12T03:10:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.686+0000 7f02bd5c4740 1 -- start start 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 0x55d1361f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d13614c100 con 0x55d1361f1570 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1360f41b0 con 0x55d1361e63b0 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.688+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1361480c0 con 0x55d1361e6cd0 2024-09-12T03:10:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.689+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.689+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38470/0 (socket says 172.21.15.57:38470) 2024-09-12T03:10:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.689+0000 7f02b3400640 1 -- 172.21.15.57:0/542625392 learned_addr learned my addr 172.21.15.57:0/542625392 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.689+0000 7f02b3e00640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02b3400640 1 -- 172.21.15.57:0/542625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 msgr2=0x55d1361f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02abe00640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 0x55d1361f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02b3400640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 0x55d1361f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02b3400640 1 -- 172.21.15.57:0/542625392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 msgr2=0x55d1361f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02b3400640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.690+0000 7f02b3400640 1 -- 172.21.15.57:0/542625392 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d135e5f210 con 0x55d1361e63b0 2024-09-12T03:10:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.691+0000 7f02abe00640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 0x55d1361f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.692+0000 7f02b3400640 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012480 tx=0x7f02a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed510dba1377eb54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.693+0000 7f02a8c00640 1 -- 172.21.15.57:0/542625392 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a401f620 con 0x55d1361e63b0 2024-09-12T03:10:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.694+0000 7f02a8c00640 1 -- 172.21.15.57:0/542625392 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a401fca0 con 0x55d1361e63b0 2024-09-12T03:10:42.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.694+0000 7f02a8c00640 1 -- 172.21.15.57:0/542625392 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4027d70 con 0x55d1361e63b0 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 -- 172.21.15.57:0/542625392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 msgr2=0x55d1361e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012480 tx=0x7f02a4014000 comp rx=0 tx=0).stop 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 -- 172.21.15.57:0/542625392 shutdown_connections 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e6cd0 0x55d1361f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.695+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/542625392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361e6790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.696+0000 7f02bd5c4740 1 -- 172.21.15.57:0/542625392 >> 172.21.15.57:0/542625392 conn(0x55d1361e2920 msgr2=0x55d1361e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.697+0000 7f02bd5c4740 1 -- 172.21.15.57:0/542625392 shutdown_connections 2024-09-12T03:10:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.697+0000 7f02bd5c4740 1 -- 172.21.15.57:0/542625392 wait complete. 2024-09-12T03:10:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.698+0000 7f02bd5c4740 1 -- start start 2024-09-12T03:10:42.698 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.698+0000 7f02bd5c4740 1 Processor -- start 2024-09-12T03:10:42.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 0x55d1361ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 0x55d1361eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55d1361f4050 con 0x55d1361f1570 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55d1360d5090 con 0x55d1361e6cd0 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.699+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d1361f41d0 con 0x55d1361e63b0 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.700+0000 7f02bd5c4740 10 client.0 _ll_register_callbacks cb 0x55d136129f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.700+0000 7f02abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 0x55d1361eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.700+0000 7f02abe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 0x55d1361eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38478/0 (socket says 172.21.15.57:38478) 2024-09-12T03:10:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.701+0000 7f02b3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 0x55d1361ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.701+0000 7f02abe00640 1 -- 172.21.15.57:0/4252019588 learned_addr learned my addr 172.21.15.57:0/4252019588 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.701+0000 7f02b3e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3400640 1 -- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 msgr2=0x55d1361eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3400640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 0x55d1361eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3400640 1 -- 172.21.15.57:0/4252019588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 msgr2=0x55d1361eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3400640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3400640 1 -- 172.21.15.57:0/4252019588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d135f2b620 con 0x55d1361e63b0 2024-09-12T03:10:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.702+0000 7f02b3e00640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.706+0000 7f02b3400640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 0x55d1361ee530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012450 tx=0x7f02a4028f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57b4104e7e9d653 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.706+0000 7f02a1600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4038060 con 0x55d1361e63b0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f02a4005040 con 0x55d1361e63b0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02a401fca0 con 0x55d1361e63b0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02a4005260 con 0x55d1361e63b0 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02bd5c4740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.707+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1361ffe80 con 0x55d1361e63b0 2024-09-12T03:10:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.708+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a401fca0 con 0x55d1361e63b0 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.709+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f02a401f420 con 0x55d1361e63b0 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.709+0000 7f02bd5c4740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.709+0000 7f02bd5c4740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.709+0000 7f02bd5c4740 20 client.25603 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.709+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d1362015c0 0x55d1362039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.710+0000 7f02bd5c4740 4 client.25603 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.710+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d136159830 con 0x55d1362015c0 2024-09-12T03:10:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.710+0000 7f02abe00640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d1362015c0 0x55d1362039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.711+0000 7f02abe00640 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d1362015c0 0x55d1362039b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9d73f623ddc76e server_cookie=99526de30011556e in_seq=0 out_seq=0 2024-09-12T03:10:42.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.712+0000 7f02a1600640 10 client.25603 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.714+0000 7f02a1600640 1 -- 172.21.15.57:0/4252019588 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d136159830 con 0x55d1362015c0 2024-09-12T03:10:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.714+0000 7f02a1600640 10 client.25603 handle_command_reply: tid=0 2024-09-12T03:10:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.714+0000 7f02bd5c4740 1 client.25603 shutdown 2024-09-12T03:10:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 msgr2=0x55d1361ee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 0x55d1361ee530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012450 tx=0x7f02a4028f80 comp rx=0 tx=0).stop 2024-09-12T03:10:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 shutdown_connections 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d1362015c0 0x55d1362039b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361f1570 0x55d1361eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e6cd0 0x55d1361eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/4252019588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361e63b0 0x55d1361ee530 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.715+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 >> 172.21.15.57:0/4252019588 conn(0x55d1361e2920 msgr2=0x55d1361e8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.716+0000 7f02a1600640 0 client.25603 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.717+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 shutdown_connections 2024-09-12T03:10:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.717+0000 7f02bd5c4740 1 -- 172.21.15.57:0/4252019588 wait complete. 2024-09-12T03:10:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.719+0000 7f02bd5c4740 20 client.25603 trim_cache size 0 max 16384 2024-09-12T03:10:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.749+0000 7f02bd5c4740 1 Processor -- start 2024-09-12T03:10:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.749+0000 7f02bd5c4740 1 -- start start 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0290012970 con 0x55d1361e6cd0 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0290010240 con 0x55d1361e63b0 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f029000e050 con 0x55d1361f1570 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:59132/0 (socket says 172.21.15.57:59132) 2024-09-12T03:10:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.750+0000 7f02b3e00640 1 -- 172.21.15.57:0/1606126970 learned_addr learned my addr 172.21.15.57:0/1606126970 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02abe00640 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 -- 172.21.15.57:0/1606126970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 msgr2=0x55d1361fccf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361fccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 -- 172.21.15.57:0/1606126970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 msgr2=0x55d136206410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 -- 172.21.15.57:0/1606126970 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1361e61e0 con 0x55d1361f1570 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02abe00640 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02b3e00640 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02ac02ae70 tx=0x7f02ac009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9785a5593040ff57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.751+0000 7f02a3e00640 1 -- 172.21.15.57:0/1606126970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac034040 con 0x55d1361f1570 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.752+0000 7f02a3e00640 1 -- 172.21.15.57:0/1606126970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ac003e50 con 0x55d1361f1570 2024-09-12T03:10:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.752+0000 7f02a3e00640 1 -- 172.21.15.57:0/1606126970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac004540 con 0x55d1361f1570 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- 172.21.15.57:0/1606126970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 msgr2=0x55d136206950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f02ac02ae70 tx=0x7f02ac009940 comp rx=0 tx=0).stop 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- 172.21.15.57:0/1606126970 shutdown_connections 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136206950 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/1606126970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d1361fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- 172.21.15.57:0/1606126970 >> 172.21.15.57:0/1606126970 conn(0x55d1361e2920 msgr2=0x55d1361fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- 172.21.15.57:0/1606126970 shutdown_connections 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- 172.21.15.57:0/1606126970 wait complete. 2024-09-12T03:10:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 -- start start 2024-09-12T03:10:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.753+0000 7f02bd5c4740 1 Processor -- start 2024-09-12T03:10:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136203a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f029000e700 con 0x55d1361e6cd0 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f029000fb70 con 0x55d1361e63b0 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f029000edd0 con 0x55d1361f1570 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02bd5c4740 10 client.0 _ll_register_callbacks cb 0x7f02a4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136203a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38502/0 (socket says 172.21.15.57:38502) 2024-09-12T03:10:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 -- 172.21.15.57:0/781136906 learned_addr learned my addr 172.21.15.57:0/781136906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02abe00640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 -- 172.21.15.57:0/781136906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 msgr2=0x55d136203a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136203a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 -- 172.21.15.57:0/781136906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 msgr2=0x55d136206b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02b3400640 1 -- 172.21.15.57:0/781136906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d136204270 con 0x55d1361e63b0 2024-09-12T03:10:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.754+0000 7f02abe00640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02b3400640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012d10 tx=0x7f02a401fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b4f520b1bea07e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02a1600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4042080 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02ac003e50 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f02a4005500 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02a404d050 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02bd5c4740 10 client.25583 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.755+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d13626ba00 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02ac003e50 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f02a401f420 con 0x55d1361e63b0 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 10 client.25583 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 20 client.25583 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d13626bed0 0x55d13626e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 4 client.25583 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:10:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d136153b80 con 0x55d13626bed0 2024-09-12T03:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.756+0000 7f02abe00640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d13626bed0 0x55d13626e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.757+0000 7f02abe00640 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d13626bed0 0x55d13626e2c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae9c97009ac4d79 server_cookie=8c555bafbe99aab8 in_seq=0 out_seq=0 2024-09-12T03:10:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.757+0000 7f02a1600640 10 client.25583 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.773+0000 7f02a1600640 1 -- 172.21.15.57:0/781136906 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d136153b80 con 0x55d13626bed0 2024-09-12T03:10:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.773+0000 7f02a1600640 10 client.25583 handle_command_reply: tid=0 2024-09-12T03:10:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.774+0000 7f02bd5c4740 1 client.25583 shutdown 2024-09-12T03:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 msgr2=0x55d136206600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02a4012d10 tx=0x7f02a401fc30 comp rx=0 tx=0).stop 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 shutdown_connections 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55d13626bed0 0x55d13626e2c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d1361f1570 0x55d136203a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55d1361e6cd0 0x55d136206b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 --2- 172.21.15.57:0/781136906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d1361e63b0 0x55d136206600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 >> 172.21.15.57:0/781136906 conn(0x55d1361e2920 msgr2=0x55d1361fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.776+0000 7f02a1600640 0 client.25583 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.777+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 shutdown_connections 2024-09-12T03:10:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.777+0000 7f02bd5c4740 1 -- 172.21.15.57:0/781136906 wait complete. 2024-09-12T03:10:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.780+0000 7f02bd5c4740 20 client.25583 trim_cache size 0 max 16384 2024-09-12T03:10:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:10:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:10:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:10:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 msgr2=0x7f02980a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 0x7f02980a52b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009c20 tx=0x7f02a4005f90 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 msgr2=0x7f02b410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 0x7f02b410ab80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f029c005fd0 tx=0x7f029c00dde0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 shutdown_connections 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f02980a2df0 0x7f02980a52b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f02b410e0f0 0x7f02b410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02b4074f40 0x7f02b410ab80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.781+0000 7f02b9000640 1 --2- 172.21.15.57:0/3454097071 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02b4074620 0x7f02b410dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.782+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 >> 172.21.15.57:0/3454097071 conn(0x7f02b40fd5d0 msgr2=0x7f02b40fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.782+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 shutdown_connections 2024-09-12T03:10:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:42.782+0000 7f02b9000640 1 -- 172.21.15.57:0/3454097071 wait complete. 2024-09-12T03:10:43.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:43 smithi057 ceph-mon[30361]: pgmap v146: 129 pgs: 129 active+clean; 31 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 18 KiB/s rd, 252 MiB/s wr, 1.92k op/s 2024-09-12T03:10:43.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:43 smithi204 ceph-mon[35013]: pgmap v146: 129 pgs: 129 active+clean; 31 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 18 KiB/s rd, 252 MiB/s wr, 1.92k op/s 2024-09-12T03:10:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:43 smithi181 ceph-mon[35320]: pgmap v146: 129 pgs: 129 active+clean; 31 GiB data, 64 GiB used, 1008 GiB / 1.0 TiB avail; 18 KiB/s rd, 252 MiB/s wr, 1.92k op/s 2024-09-12T03:10:45.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:45 smithi057 ceph-mon[30361]: pgmap v147: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 15 KiB/s rd, 206 MiB/s wr, 1.44k op/s 2024-09-12T03:10:45.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:45 smithi204 ceph-mon[35013]: pgmap v147: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 15 KiB/s rd, 206 MiB/s wr, 1.44k op/s 2024-09-12T03:10:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:45 smithi181 ceph-mon[35320]: pgmap v147: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 15 KiB/s rd, 206 MiB/s wr, 1.44k op/s 2024-09-12T03:10:47.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:47 smithi057 ceph-mon[30361]: pgmap v148: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 16 KiB/s rd, 219 MiB/s wr, 1.47k op/s 2024-09-12T03:10:47.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:47 smithi204 ceph-mon[35013]: pgmap v148: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 16 KiB/s rd, 219 MiB/s wr, 1.47k op/s 2024-09-12T03:10:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:47 smithi181 ceph-mon[35320]: pgmap v148: 129 pgs: 129 active+clean; 31 GiB data, 65 GiB used, 1007 GiB / 1.0 TiB avail; 16 KiB/s rd, 219 MiB/s wr, 1.47k op/s 2024-09-12T03:10:49.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:49 smithi057 ceph-mon[30361]: pgmap v149: 129 pgs: 129 active+clean; 32 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 23 KiB/s rd, 270 MiB/s wr, 1.63k op/s 2024-09-12T03:10:49.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:49 smithi204 ceph-mon[35013]: pgmap v149: 129 pgs: 129 active+clean; 32 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 23 KiB/s rd, 270 MiB/s wr, 1.63k op/s 2024-09-12T03:10:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:49 smithi181 ceph-mon[35320]: pgmap v149: 129 pgs: 129 active+clean; 32 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 23 KiB/s rd, 270 MiB/s wr, 1.63k op/s 2024-09-12T03:10:51.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:51 smithi057 ceph-mon[30361]: pgmap v150: 129 pgs: 129 active+clean; 32 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 1.27k op/s 2024-09-12T03:10:51.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:51 smithi204 ceph-mon[35013]: pgmap v150: 129 pgs: 129 active+clean; 32 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 1.27k op/s 2024-09-12T03:10:51.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:51 smithi181 ceph-mon[35320]: pgmap v150: 129 pgs: 129 active+clean; 32 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 13 KiB/s rd, 232 MiB/s wr, 1.27k op/s 2024-09-12T03:10:53.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:53 smithi057 ceph-mon[30361]: pgmap v151: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 16 KiB/s rd, 265 MiB/s wr, 1.35k op/s 2024-09-12T03:10:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:53 smithi204 ceph-mon[35013]: pgmap v151: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 16 KiB/s rd, 265 MiB/s wr, 1.35k op/s 2024-09-12T03:10:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:53 smithi181 ceph-mon[35320]: pgmap v151: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 16 KiB/s rd, 265 MiB/s wr, 1.35k op/s 2024-09-12T03:10:55.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:55 smithi057 ceph-mon[30361]: pgmap v152: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.02k op/s 2024-09-12T03:10:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:55 smithi204 ceph-mon[35013]: pgmap v152: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.02k op/s 2024-09-12T03:10:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:55 smithi181 ceph-mon[35320]: pgmap v152: 129 pgs: 129 active+clean; 33 GiB data, 69 GiB used, 1004 GiB / 1.0 TiB avail; 12 KiB/s rd, 213 MiB/s wr, 1.02k op/s 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.479+0000 7f0098600640 1 -- 172.21.15.57:0/1273475936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00901138b0 msgr2=0x7f0090115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.479+0000 7f0098600640 1 --2- 172.21.15.57:0/1273475936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00901138b0 0x7f0090115ca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f008800f0a0 tx=0x7f008802a2e0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 -- 172.21.15.57:0/1273475936 shutdown_connections 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 --2- 172.21.15.57:0/1273475936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00901138b0 0x7f0090115ca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 --2- 172.21.15.57:0/1273475936 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f0090113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 --2- 172.21.15.57:0/1273475936 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f009010a300 0x7f009010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 -- 172.21.15.57:0/1273475936 >> 172.21.15.57:0/1273475936 conn(0x7f009006ce00 msgr2=0x7f009006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.480+0000 7f0098600640 1 -- 172.21.15.57:0/1273475936 shutdown_connections 2024-09-12T03:10:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.481+0000 7f0098600640 1 -- 172.21.15.57:0/1273475936 wait complete. 2024-09-12T03:10:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.486+0000 7f0098600640 1 Processor -- start 2024-09-12T03:10:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.487+0000 7f0098600640 1 -- start start 2024-09-12T03:10:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.490+0000 7f0098600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 0x7f00901a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.490+0000 7f0098600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.490+0000 7f0098600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 0x7f009019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.491+0000 7f0098600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0090118460 con 0x7f00901138b0 2024-09-12T03:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55680/0 (socket says 172.21.15.57:55680) 2024-09-12T03:10:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095200640 1 -- 172.21.15.57:0/3726517006 learned_addr learned my addr 172.21.15.57:0/3726517006 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00901182e0 con 0x7f009010a300 2024-09-12T03:10:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00901185e0 con 0x7f009010acb0 2024-09-12T03:10:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0096600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 0x7f009019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095200640 1 -- 172.21.15.57:0/3726517006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 msgr2=0x7f00901a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095c00640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 0x7f00901a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.492+0000 7f0095200640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 0x7f00901a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.493+0000 7f0095200640 1 -- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 msgr2=0x7f009019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.493+0000 7f0095200640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 0x7f009019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.493+0000 7f0095200640 1 -- 172.21.15.57:0/3726517006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009019eda0 con 0x7f009010acb0 2024-09-12T03:10:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.493+0000 7f0096600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 0x7f009019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.494+0000 7f0095c00640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 0x7f00901a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:10:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.496+0000 7f0095200640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f008000ff50 tx=0x7f00800134e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.500+0000 7f008e000640 1 -- 172.21.15.57:0/3726517006 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080004220 con 0x7f009010acb0 2024-09-12T03:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.500+0000 7f008e000640 1 -- 172.21.15.57:0/3726517006 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00800043c0 con 0x7f009010acb0 2024-09-12T03:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.500+0000 7f008e000640 1 -- 172.21.15.57:0/3726517006 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008001da60 con 0x7f009010acb0 2024-09-12T03:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.500+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009019f090 con 0x7f009010acb0 2024-09-12T03:10:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.500+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901ab330 con 0x7f009010acb0 2024-09-12T03:10:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.510+0000 7f008e000640 1 -- 172.21.15.57:0/3726517006 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f008001dc00 con 0x7f009010acb0 2024-09-12T03:10:56.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.518+0000 7f008e000640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 0x7f00740a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.518+0000 7f009b621740 1 Processor -- start 2024-09-12T03:10:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.519+0000 7f009b621740 1 -- start start 2024-09-12T03:10:56.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.519+0000 7f008e000640 1 -- 172.21.15.57:0/3726517006 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f00800d75f0 con 0x7f009010acb0 2024-09-12T03:10:56.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.520+0000 7f009b621740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 0x5586dbfb95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f0095c00640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 0x7f00740a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f009b621740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f009b621740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f009b621740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5586dbf1f100 con 0x5586dbfc43a0 2024-09-12T03:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f009b621740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5586dbec71b0 con 0x5586dbfb91e0 2024-09-12T03:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.521+0000 7f009b621740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586dbf1b0c0 con 0x5586dbfb9b00 2024-09-12T03:10:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.523+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.523+0000 7f0095c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 0x5586dbfb95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.523+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55692/0 (socket says 172.21.15.57:55692) 2024-09-12T03:10:56.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.523+0000 7f0095200640 1 -- 172.21.15.57:0/1623844980 learned_addr learned my addr 172.21.15.57:0/1623844980 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.524+0000 7f0095c00640 1 -- 172.21.15.57:0/1623844980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 msgr2=0x5586dbfc3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.524+0000 7f0095c00640 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.524+0000 7f0095c00640 1 -- 172.21.15.57:0/1623844980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 msgr2=0x5586dbfc6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.524+0000 7f0095c00640 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.524+0000 7f0095c00640 1 -- 172.21.15.57:0/1623844980 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586dbc32210 con 0x5586dbfb91e0 2024-09-12T03:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.525+0000 7f0095c00640 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 0x5586dbfb95c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f007c014a10 tx=0x7f007c014ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8f7e181b0366ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.526+0000 7f0095c00640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 0x7f00740a5490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f007c008fd0 tx=0x7f007c00f000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.528+0000 7f0086a00640 1 -- 172.21.15.57:0/1623844980 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c024070 con 0x5586dbfb91e0 2024-09-12T03:10:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.528+0000 7f0086a00640 1 -- 172.21.15.57:0/1623844980 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007c0086e0 con 0x5586dbfb91e0 2024-09-12T03:10:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.528+0000 7f0086a00640 1 -- 172.21.15.57:0/1623844980 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c0126b0 con 0x5586dbfb91e0 2024-09-12T03:10:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.529+0000 7f0095200640 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 -- 172.21.15.57:0/1623844980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 msgr2=0x5586dbfb95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 0x5586dbfb95c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f007c014a10 tx=0x7f007c014ed0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 -- 172.21.15.57:0/1623844980 shutdown_connections 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 --2- 172.21.15.57:0/1623844980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb91e0 0x5586dbfb95c0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f007c014a10 tx=0x7f007c014ed0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 -- 172.21.15.57:0/1623844980 >> 172.21.15.57:0/1623844980 conn(0x5586dbfb5750 msgr2=0x5586dbfb5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 -- 172.21.15.57:0/1623844980 shutdown_connections 2024-09-12T03:10:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.530+0000 7f009b621740 1 -- 172.21.15.57:0/1623844980 wait complete. 2024-09-12T03:10:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.532+0000 7f009b621740 1 -- start start 2024-09-12T03:10:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.534+0000 7f009b621740 1 Processor -- start 2024-09-12T03:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.535+0000 7f009b621740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.535+0000 7f009b621740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.535+0000 7f0095c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.536 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.535+0000 7f0095c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55700/0 (socket says 172.21.15.57:55700) 2024-09-12T03:10:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f0095c00640 1 -- 172.21.15.57:0/1710420576 learned_addr learned my addr 172.21.15.57:0/1710420576 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f0095200640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc1f40 0x5586dbfd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5586dbfc6f70 con 0x5586dbfc43a0 2024-09-12T03:10:56.537 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5586dbea8090 con 0x5586dbfc1f40 2024-09-12T03:10:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586dbfc70f0 con 0x5586dbfb9b00 2024-09-12T03:10:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.536+0000 7f0096600640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc1f40 0x5586dbfd0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095200640 1 -- 172.21.15.57:0/1710420576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 msgr2=0x5586dbfc14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095200640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095200640 1 -- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc1f40 msgr2=0x5586dbfd0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095200640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc1f40 0x5586dbfd0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095200640 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586dbcfe620 con 0x5586dbfc43a0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f0095c00640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f009b621740 10 client.0 _ll_register_callbacks cb 0x5586dbefcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.537+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586dbf2c830 con 0x5586dbfc43a0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.538+0000 7f0095200640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc1a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00800a0c60 tx=0x7f008001d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7491b57e44d08a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.538+0000 7f007b400640 10 client.15480 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.538+0000 7f007b400640 1 -- 172.21.15.57:0/1710420576 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800bd070 con 0x5586dbfc43a0 2024-09-12T03:10:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.539+0000 7f007b400640 1 -- 172.21.15.57:0/1710420576 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f00800d7390 con 0x5586dbfc43a0 2024-09-12T03:10:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.539+0000 7f007b400640 1 -- 172.21.15.57:0/1710420576 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00800d4dc0 con 0x5586dbfc43a0 2024-09-12T03:10:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.539+0000 7f009b621740 10 client.15480 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.540+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5586dbfd3120 con 0x5586dbfc43a0 2024-09-12T03:10:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f007b400640 1 -- 172.21.15.57:0/1710420576 <== mon.0 v2:172.21.15.57:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f00800c0030 con 0x5586dbfc43a0 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 20 client.15480 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dbfd4590 0x5586dbfd6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 4 client.15480 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:10:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.542+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586dbf26b80 con 0x5586dbfd4590 2024-09-12T03:10:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.543+0000 7f0095c00640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dbfd4590 0x5586dbfd6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.544+0000 7f0095c00640 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dbfd4590 0x5586dbfd6a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c715458b7211f3 server_cookie=a4f56306c09da9c6 in_seq=0 out_seq=0 2024-09-12T03:10:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.544+0000 7f007b400640 10 client.15480 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.545+0000 7f007b400640 1 -- 172.21.15.57:0/1710420576 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586dbf26b80 con 0x5586dbfd4590 2024-09-12T03:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.545+0000 7f007b400640 10 client.15480 handle_command_reply: tid=0 2024-09-12T03:10:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.545+0000 7f009b621740 1 client.15480 shutdown 2024-09-12T03:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.545+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 msgr2=0x5586dbfc1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.545+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc1a00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f00800a0c60 tx=0x7f008001d4c0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 shutdown_connections 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dbfd4590 0x5586dbfd6a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc1f40 0x5586dbfd0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dbfc1a00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 --2- 172.21.15.57:0/1710420576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfb9b00 0x5586dbfc14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 >> 172.21.15.57:0/1710420576 conn(0x5586dbfb5750 msgr2=0x5586dbfbb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.546+0000 7f007b400640 0 client.15480 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.548+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 shutdown_connections 2024-09-12T03:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.548+0000 7f009b621740 1 -- 172.21.15.57:0/1710420576 wait complete. 2024-09-12T03:10:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.548+0000 7f009b621740 20 client.15480 trim_cache size 0 max 16384 2024-09-12T03:10:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.569+0000 7f009b621740 1 Processor -- start 2024-09-12T03:10:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.569+0000 7f009b621740 1 -- start start 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfb9b00 0x5586dbfd0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 0x5586dbfbccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f00700127d0 con 0x5586dbfb9b00 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f00700100a0 con 0x5586dbfc43a0 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f009b621740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f007000def0 con 0x5586dbfc1f40 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55706/0 (socket says 172.21.15.57:55706) 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 -- 172.21.15.57:0/227280592 learned_addr learned my addr 172.21.15.57:0/227280592 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0096600640 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 0x5586dbfbccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 -- 172.21.15.57:0/227280592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 msgr2=0x5586dbfbccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 0x5586dbfbccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 -- 172.21.15.57:0/227280592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfb9b00 msgr2=0x5586dbfd0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfb9b00 0x5586dbfd0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.570+0000 7f0095200640 1 -- 172.21.15.57:0/227280592 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586dbfbf9f0 con 0x5586dbfc1f40 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.571+0000 7f0096600640 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 0x5586dbfbccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.571+0000 7f0095200640 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f00800a11d0 tx=0x7f00800ba000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56e907c05e7ff23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.571+0000 7f0086000640 1 -- 172.21.15.57:0/227280592 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800bd050 con 0x5586dbfc1f40 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.571+0000 7f0086000640 1 -- 172.21.15.57:0/227280592 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5586dbf2c830 con 0x5586dbfc1f40 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.571+0000 7f0086000640 1 -- 172.21.15.57:0/227280592 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800a5d90 con 0x5586dbfc1f40 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- 172.21.15.57:0/227280592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 msgr2=0x5586dbfbc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f00800a11d0 tx=0x7f00800ba000 comp rx=0 tx=0).stop 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- 172.21.15.57:0/227280592 shutdown_connections 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfc43a0 0x5586dbfbccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfbc780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 --2- 172.21.15.57:0/227280592 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfb9b00 0x5586dbfd0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- 172.21.15.57:0/227280592 >> 172.21.15.57:0/227280592 conn(0x5586dbfb5750 msgr2=0x5586dbfc1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- 172.21.15.57:0/227280592 shutdown_connections 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- 172.21.15.57:0/227280592 wait complete. 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.572+0000 7f009b621740 1 -- start start 2024-09-12T03:10:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 Processor -- start 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfc7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dc03d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f007000e560 con 0x5586dbfc43a0 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f007000f9d0 con 0x5586dbfb9b00 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f007000ec30 con 0x5586dbfc1f40 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f009b621740 10 client.0 _ll_register_callbacks cb 0x7f00800a4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f0095c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f0095c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51578/0 (socket says 172.21.15.57:51578) 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f0095c00640 1 -- 172.21.15.57:0/410668829 learned_addr learned my addr 172.21.15.57:0/410668829 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f0095200640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfc7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.573+0000 7f0096600640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dc03d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 -- 172.21.15.57:0/410668829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 msgr2=0x5586dbfc7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfc7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 -- 172.21.15.57:0/410668829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 msgr2=0x5586dc03d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dc03d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.579 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 -- 172.21.15.57:0/410668829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586dc03d830 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f0095c00640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f007c014a10 tx=0x7f007c008df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb7373c8e3a8dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f007b400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007c024040 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586dbf2c830 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.574+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f007c00ea90 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f007c0142e0 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 10 client.25598 fetch_fsmap learned FSMap version 30 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586dc03e9e0 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5586dbf2c830 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f007c02b930 con 0x5586dbfb9b00 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 10 client.25598 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:10:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 20 client.25598 populate_metadata read hostname 'smithi057' 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dc03ee30 0x5586dc041250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 4 client.25598 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.575+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586dbea37c0 con 0x5586dc03ee30 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.576+0000 7f0095200640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dc03ee30 0x5586dc041250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.576+0000 7f007b400640 10 client.25598 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.576+0000 7f0095200640 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dc03ee30 0x5586dc041250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2472ce0e48a162bb server_cookie=ebb241bcd92e7b89 in_seq=0 out_seq=0 2024-09-12T03:10:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f007b400640 1 -- 172.21.15.57:0/410668829 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586dbea37c0 con 0x5586dc03ee30 2024-09-12T03:10:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f007b400640 10 client.25598 handle_command_reply: tid=0 2024-09-12T03:10:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 client.25598 shutdown 2024-09-12T03:10:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 msgr2=0x5586dbfc73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f007c014a10 tx=0x7f007c008df0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 shutdown_connections 2024-09-12T03:10:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5586dc03ee30 0x5586dc041250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5586dbfc43a0 0x5586dc03d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586dbfc1f40 0x5586dbfc7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 --2- 172.21.15.57:0/410668829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5586dbfb9b00 0x5586dbfc73f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f007b400640 0 client.25598 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 >> 172.21.15.57:0/410668829 conn(0x5586dbfb5750 msgr2=0x5586dbfc1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 shutdown_connections 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.587+0000 7f009b621740 1 -- 172.21.15.57:0/410668829 wait complete. 2024-09-12T03:10:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.588+0000 7f009b621740 20 client.25598 trim_cache size 0 max 16384 2024-09-12T03:10:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:10:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:10:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:10:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:10:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 msgr2=0x7f00740a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 0x7f00740a5490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f007c008fd0 tx=0x7f007c00f000 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 msgr2=0x7f00901a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f008000ff50 tx=0x7f00800134e0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 shutdown_connections 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f00740a2fd0 0x7f00740a5490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f00901138b0 0x7f009019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f009010acb0 0x7f00901a49b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 --2- 172.21.15.57:0/3726517006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f009010a300 0x7f00901a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.589+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 >> 172.21.15.57:0/3726517006 conn(0x7f009006ce00 msgr2=0x7f00900707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.590+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 shutdown_connections 2024-09-12T03:10:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:10:56.590+0000 7f0098600640 1 -- 172.21.15.57:0/3726517006 wait complete. 2024-09-12T03:10:57.871 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:57 smithi057 ceph-mon[30361]: pgmap v153: 129 pgs: 129 active+clean; 33 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-12T03:10:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:57 smithi204 ceph-mon[35013]: pgmap v153: 129 pgs: 129 active+clean; 33 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-12T03:10:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:57 smithi181 ceph-mon[35320]: pgmap v153: 129 pgs: 129 active+clean; 33 GiB data, 70 GiB used, 1003 GiB / 1.0 TiB avail; 13 KiB/s rd, 227 MiB/s wr, 1.09k op/s 2024-09-12T03:10:59.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:10:59 smithi057 ceph-mon[30361]: pgmap v154: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 1.32k op/s 2024-09-12T03:10:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:10:59 smithi204 ceph-mon[35013]: pgmap v154: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 1.32k op/s 2024-09-12T03:10:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:10:59 smithi181 ceph-mon[35320]: pgmap v154: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1002 GiB / 1.0 TiB avail; 18 KiB/s rd, 276 MiB/s wr, 1.32k op/s 2024-09-12T03:11:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:01 smithi204 ceph-mon[35013]: pgmap v155: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.08k op/s 2024-09-12T03:11:01.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:01 smithi181 ceph-mon[35320]: pgmap v155: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.08k op/s 2024-09-12T03:11:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:01 smithi057 ceph-mon[30361]: pgmap v155: 129 pgs: 129 active+clean; 34 GiB data, 71 GiB used, 1001 GiB / 1.0 TiB avail; 11 KiB/s rd, 229 MiB/s wr, 1.08k op/s 2024-09-12T03:11:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:03 smithi204 ceph-mon[35013]: pgmap v156: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 17 KiB/s rd, 268 MiB/s wr, 1.27k op/s 2024-09-12T03:11:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:03 smithi181 ceph-mon[35320]: pgmap v156: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 17 KiB/s rd, 268 MiB/s wr, 1.27k op/s 2024-09-12T03:11:04.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:03 smithi057 ceph-mon[30361]: pgmap v156: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 17 KiB/s rd, 268 MiB/s wr, 1.27k op/s 2024-09-12T03:11:05.960 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:05 smithi181 ceph-mon[35320]: pgmap v157: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 13 KiB/s rd, 217 MiB/s wr, 1.05k op/s 2024-09-12T03:11:06.124 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:05 smithi057 ceph-mon[30361]: pgmap v157: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 13 KiB/s rd, 217 MiB/s wr, 1.05k op/s 2024-09-12T03:11:06.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:05 smithi204 ceph-mon[35013]: pgmap v157: 129 pgs: 129 active+clean; 35 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 13 KiB/s rd, 217 MiB/s wr, 1.05k op/s 2024-09-12T03:11:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:07 smithi181 ceph-mon[35320]: pgmap v158: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 1.15k op/s 2024-09-12T03:11:08.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:07 smithi057 ceph-mon[30361]: pgmap v158: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 1.15k op/s 2024-09-12T03:11:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:07 smithi204 ceph-mon[35013]: pgmap v158: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 22 KiB/s rd, 231 MiB/s wr, 1.15k op/s 2024-09-12T03:11:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:09 smithi181 ceph-mon[35320]: pgmap v159: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 28 KiB/s rd, 279 MiB/s wr, 1.47k op/s 2024-09-12T03:11:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:09 smithi057 ceph-mon[30361]: pgmap v159: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 28 KiB/s rd, 279 MiB/s wr, 1.47k op/s 2024-09-12T03:11:10.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:09 smithi204 ceph-mon[35013]: pgmap v159: 129 pgs: 129 active+clean; 35 GiB data, 73 GiB used, 999 GiB / 1.0 TiB avail; 28 KiB/s rd, 279 MiB/s wr, 1.47k op/s 2024-09-12T03:11:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.657+0000 7f6633200640 1 -- 172.21.15.57:0/3385280823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6634102ae0 msgr2=0x7f6634102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.657+0000 7f6633200640 1 --2- 172.21.15.57:0/3385280823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6634102ae0 0x7f6634102ec0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6624009900 tx=0x7f662402a480 comp rx=0 tx=0).stop 2024-09-12T03:11:10.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 -- 172.21.15.57:0/3385280823 shutdown_connections 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 --2- 172.21.15.57:0/3385280823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f663410bf00 0x7f663410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 --2- 172.21.15.57:0/3385280823 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f663410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 --2- 172.21.15.57:0/3385280823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6634102ae0 0x7f6634102ec0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 -- 172.21.15.57:0/3385280823 >> 172.21.15.57:0/3385280823 conn(0x7f66340777c0 msgr2=0x7f66340feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 -- 172.21.15.57:0/3385280823 shutdown_connections 2024-09-12T03:11:10.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.658+0000 7f6633200640 1 -- 172.21.15.57:0/3385280823 wait complete. 2024-09-12T03:11:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.659+0000 7f6633200640 1 Processor -- start 2024-09-12T03:11:10.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 -- start start 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f66341988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663410bf00 0x7f663419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6634111d60 con 0x7f6634103400 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6634111be0 con 0x7f6634102ae0 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.660+0000 7f6633200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6634111ee0 con 0x7f663410bf00 2024-09-12T03:11:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6632800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663410bf00 0x7f663419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46002/0 (socket says 172.21.15.57:46002) 2024-09-12T03:11:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 -- 172.21.15.57:0/1491094522 learned_addr learned my addr 172.21.15.57:0/1491094522 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 -- 172.21.15.57:0/1491094522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663410bf00 msgr2=0x7f663419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663410bf00 0x7f663419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631e00640 1 -- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 msgr2=0x7f66341988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.661+0000 7f6631400640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f66341988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f6631e00640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f66341988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f6631e00640 1 -- 172.21.15.57:0/1491094522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663419d350 con 0x7f6634102ae0 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f6631400640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f66341988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f6631e00640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f662402a9f0 tx=0x7f6624003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f662aa00640 1 -- 172.21.15.57:0/1491094522 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662403f070 con 0x7f6634102ae0 2024-09-12T03:11:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f662aa00640 1 -- 172.21.15.57:0/1491094522 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6624031c90 con 0x7f6634102ae0 2024-09-12T03:11:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f662aa00640 1 -- 172.21.15.57:0/1491094522 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624035030 con 0x7f6634102ae0 2024-09-12T03:11:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.662+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663419d5e0 con 0x7f6634102ae0 2024-09-12T03:11:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.663+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6634100660 con 0x7f6634102ae0 2024-09-12T03:11:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.671+0000 7f662aa00640 1 -- 172.21.15.57:0/1491094522 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f6624042070 con 0x7f6634102ae0 2024-09-12T03:11:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.673+0000 7f662aa00640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 0x7f66180a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.673+0000 7f662aa00640 1 -- 172.21.15.57:0/1491094522 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f66240effd0 con 0x7f6634102ae0 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.674+0000 7f663badb740 1 Processor -- start 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.674+0000 7f6631400640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 0x7f66180a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.674+0000 7f663badb740 1 -- start start 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fbe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fc9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fcb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558602f24100 con 0x558602fbecd0 2024-09-12T03:11:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558602ecc1b0 con 0x558602fc9570 2024-09-12T03:11:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f663badb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558602f200c0 con 0x558602fbe3b0 2024-09-12T03:11:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.675+0000 7f6631400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fc9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.676+0000 7f6631400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fc9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42686/0 (socket says 172.21.15.57:42686) 2024-09-12T03:11:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.676+0000 7f6631400640 1 -- 172.21.15.57:0/1399352977 learned_addr learned my addr 172.21.15.57:0/1399352977 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.676+0000 7f6632800640 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fcb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.677+0000 7f6632800640 1 -- 172.21.15.57:0/1399352977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 msgr2=0x558602fbe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.680 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.677+0000 7f6632800640 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fbe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.679+0000 7f6632800640 1 -- 172.21.15.57:0/1399352977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 msgr2=0x558602fc9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.679+0000 7f6632800640 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fc9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.679+0000 7f6632800640 1 -- 172.21.15.57:0/1399352977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558602c37210 con 0x558602fc9570 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.679+0000 7f6632800640 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fcb960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f662c009dd0 tx=0x7f662c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719013e40dd528a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.680+0000 7f661f000640 1 -- 172.21.15.57:0/1399352977 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c004b20 con 0x558602fc9570 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.680+0000 7f661f000640 1 -- 172.21.15.57:0/1399352977 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c017070 con 0x558602fc9570 2024-09-12T03:11:10.681 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.680+0000 7f661f000640 1 -- 172.21.15.57:0/1399352977 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0117c0 con 0x558602fc9570 2024-09-12T03:11:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.682+0000 7f6631400640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 0x7f66180a5280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6620013600 tx=0x7f662000b530 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.682+0000 7f663badb740 1 -- 172.21.15.57:0/1399352977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 msgr2=0x558602fcb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.682+0000 7f663badb740 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fcb960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f662c009dd0 tx=0x7f662c00f730 comp rx=0 tx=0).stop 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 -- 172.21.15.57:0/1399352977 shutdown_connections 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fcb960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fc9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 --2- 172.21.15.57:0/1399352977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fbe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.683 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 -- 172.21.15.57:0/1399352977 >> 172.21.15.57:0/1399352977 conn(0x558602fba920 msgr2=0x558602fbad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 -- 172.21.15.57:0/1399352977 shutdown_connections 2024-09-12T03:11:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.683+0000 7f663badb740 1 -- 172.21.15.57:0/1399352977 wait complete. 2024-09-12T03:11:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.684+0000 7f663badb740 1 -- start start 2024-09-12T03:11:10.685 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.684+0000 7f663badb740 1 Processor -- start 2024-09-12T03:11:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 0x558602fc6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fc9570 0x558602fc6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f6631e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 0x558602fc6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f6631e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 0x558602fc6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46022/0 (socket says 172.21.15.57:46022) 2024-09-12T03:11:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f6631e00640 1 -- 172.21.15.57:0/3782634967 learned_addr learned my addr 172.21.15.57:0/3782634967 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558602fcc050 con 0x558602fc9570 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558602ead090 con 0x558602fbe3b0 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.687+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558602fcc1d0 con 0x558602fbecd0 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f663badb740 10 client.0 _ll_register_callbacks cb 0x558602f01f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f6631400640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f6631400640 1 -- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 msgr2=0x558602fc6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f6631400640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 0x558602fc6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f6631400640 1 -- 172.21.15.57:0/3782634967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fc9570 msgr2=0x558602fc6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.688+0000 7f6632800640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fc9570 0x558602fc6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.689+0000 7f6631400640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fc9570 0x558602fc6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.689+0000 7f6631400640 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558602d03620 con 0x558602fbecd0 2024-09-12T03:11:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.690+0000 7f6631400640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc6a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6620013e90 tx=0x7f66200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=818b4662b7491065 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.690+0000 7f6613400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:11:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.690+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620027060 con 0x558602fbecd0 2024-09-12T03:11:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.690+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f6620017db0 con 0x558602fbecd0 2024-09-12T03:11:10.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.691+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c017070 con 0x558602fbecd0 2024-09-12T03:11:10.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.704+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f662000f040 con 0x558602fbecd0 2024-09-12T03:11:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.704+0000 7f663badb740 10 client.25633 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.704+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558602fd7ed0 con 0x558602fbecd0 2024-09-12T03:11:10.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c017070 con 0x558602fbecd0 2024-09-12T03:11:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6620021880 con 0x558602fbecd0 2024-09-12T03:11:10.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 10 client.25633 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 20 client.25633 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558602fd93b0 0x558602fdb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 4 client.25633 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.705+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558602f31830 con 0x558602fd93b0 2024-09-12T03:11:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.706+0000 7f6631e00640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558602fd93b0 0x558602fdb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.707+0000 7f6631e00640 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558602fd93b0 0x558602fdb7a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7c79754e24981d server_cookie=3269d4513a3dcb12 in_seq=0 out_seq=0 2024-09-12T03:11:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.707+0000 7f6613400640 10 client.25633 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.708+0000 7f6613400640 1 -- 172.21.15.57:0/3782634967 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558602f31830 con 0x558602fd93b0 2024-09-12T03:11:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.708+0000 7f6613400640 10 client.25633 handle_command_reply: tid=0 2024-09-12T03:11:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.709+0000 7f663badb740 1 client.25633 shutdown 2024-09-12T03:11:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 msgr2=0x558602fc6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc6a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6620013e90 tx=0x7f66200046c0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 shutdown_connections 2024-09-12T03:11:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558602fd93b0 0x558602fdb7a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fc9570 0x558602fc6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc6a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 --2- 172.21.15.57:0/3782634967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fbe3b0 0x558602fc6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 >> 172.21.15.57:0/3782634967 conn(0x558602fba920 msgr2=0x558602fc0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.711+0000 7f6613400640 0 client.25633 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.712+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 shutdown_connections 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.712+0000 7f663badb740 1 -- 172.21.15.57:0/3782634967 wait complete. 2024-09-12T03:11:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.712+0000 7f663badb740 20 client.25633 trim_cache size 0 max 16384 2024-09-12T03:11:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.761+0000 7f663badb740 1 Processor -- start 2024-09-12T03:11:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 -- start start 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fc8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fe10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b7e00 con 0x558602fbecd0 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b56d0 con 0x558602fc9570 2024-09-12T03:11:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.766+0000 7f663badb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b3610 con 0x558602fbe3b0 2024-09-12T03:11:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.767+0000 7f6632800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.769+0000 7f6632800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46034/0 (socket says 172.21.15.57:46034) 2024-09-12T03:11:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.769+0000 7f6632800640 1 -- 172.21.15.57:0/3644272513 learned_addr learned my addr 172.21.15.57:0/3644272513 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6631400640 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fe10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 -- 172.21.15.57:0/3644272513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 msgr2=0x558602fc8390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fc8390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 -- 172.21.15.57:0/3644272513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 msgr2=0x558602fe10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fe10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 -- 172.21.15.57:0/3644272513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558602fbb5b0 con 0x558602fc9570 2024-09-12T03:11:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.770+0000 7f6632800640 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f662c000f80 tx=0x7f662c009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc66e42d9cbf074 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.771+0000 7f661e600640 1 -- 172.21.15.57:0/3644272513 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c004ba0 con 0x558602fc9570 2024-09-12T03:11:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.771+0000 7f661e600640 1 -- 172.21.15.57:0/3644272513 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c017070 con 0x558602fc9570 2024-09-12T03:11:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.773+0000 7f661e600640 1 -- 172.21.15.57:0/3644272513 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c011680 con 0x558602fc9570 2024-09-12T03:11:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 -- 172.21.15.57:0/3644272513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 msgr2=0x558602fe1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f662c000f80 tx=0x7f662c009860 comp rx=0 tx=0).stop 2024-09-12T03:11:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 -- 172.21.15.57:0/3644272513 shutdown_connections 2024-09-12T03:11:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fe1600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbecd0 0x558602fe10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 --2- 172.21.15.57:0/3644272513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbe3b0 0x558602fc8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.774+0000 7f663badb740 1 -- 172.21.15.57:0/3644272513 >> 172.21.15.57:0/3644272513 conn(0x558602fba920 msgr2=0x558602fc7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.775+0000 7f663badb740 1 -- 172.21.15.57:0/3644272513 shutdown_connections 2024-09-12T03:11:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.777+0000 7f663badb740 1 -- 172.21.15.57:0/3644272513 wait complete. 2024-09-12T03:11:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.777+0000 7f663badb740 1 -- start start 2024-09-12T03:11:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.777+0000 7f663badb740 1 Processor -- start 2024-09-12T03:11:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.779+0000 7f663badb740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 0x558602fc34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.779+0000 7f663badb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.779+0000 7f663badb740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f663badb740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b3b90 con 0x558602fbe3b0 2024-09-12T03:11:10.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f663badb740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b5000 con 0x558602fc9570 2024-09-12T03:11:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f663badb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f66341b4260 con 0x558602fbecd0 2024-09-12T03:11:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f663badb740 10 client.0 _ll_register_callbacks cb 0x7f662000b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:46046/0 (socket says 172.21.15.57:46046) 2024-09-12T03:11:10.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 -- 172.21.15.57:0/38200702 learned_addr learned my addr 172.21.15.57:0/38200702 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6631400640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6631e00640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 0x558602fc34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 -- 172.21.15.57:0/38200702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 msgr2=0x558602fc3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.780+0000 7f6632800640 1 -- 172.21.15.57:0/38200702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 msgr2=0x558602fc34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.781+0000 7f6632800640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 0x558602fc34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.781+0000 7f6632800640 1 -- 172.21.15.57:0/38200702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558603043090 con 0x558602fc9570 2024-09-12T03:11:10.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.781+0000 7f6631e00640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 0x558602fc34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.782+0000 7f6632800640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f662c025040 tx=0x7f662c004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=562d641a235ba9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.782+0000 7f6613400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:11:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.782+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c030070 con 0x558602fc9570 2024-09-12T03:11:10.784 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.783+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c017070 con 0x558602fc9570 2024-09-12T03:11:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.783+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f662c005200 con 0x558602fc9570 2024-09-12T03:11:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.785+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f662c03c050 con 0x558602fc9570 2024-09-12T03:11:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.785+0000 7f663badb740 10 client.25616 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:10.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.785+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586030442c0 con 0x558602fc9570 2024-09-12T03:11:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.788+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662c017070 con 0x558602fc9570 2024-09-12T03:11:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.788+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f662c02c430 con 0x558602fc9570 2024-09-12T03:11:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 10 client.25616 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 20 client.25616 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:10.792 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558603044740 0x558603046b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 4 client.25616 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:11:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.791+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558602f2bb80 con 0x558603044740 2024-09-12T03:11:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.792+0000 7f6631e00640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558603044740 0x558603046b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.793+0000 7f6631e00640 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558603044740 0x558603046b30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c63653031a9331 server_cookie=b36d9e8875d27232 in_seq=0 out_seq=0 2024-09-12T03:11:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.793+0000 7f6613400640 10 client.25616 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.815+0000 7f6613400640 1 -- 172.21.15.57:0/38200702 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558602f2bb80 con 0x558603044740 2024-09-12T03:11:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.815+0000 7f6613400640 10 client.25616 handle_command_reply: tid=0 2024-09-12T03:11:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.815+0000 7f663badb740 1 client.25616 shutdown 2024-09-12T03:11:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.816+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 msgr2=0x558602fc3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.816+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f662c025040 tx=0x7f662c004bd0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 shutdown_connections 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558603044740 0x558603046b30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558602fc9570 0x558602fc3f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558602fbecd0 0x558602fc3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f6613400640 0 client.25616 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 --2- 172.21.15.57:0/38200702 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558602fbe3b0 0x558602fc34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 >> 172.21.15.57:0/38200702 conn(0x558602fba920 msgr2=0x558602fc0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 shutdown_connections 2024-09-12T03:11:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.818+0000 7f663badb740 1 -- 172.21.15.57:0/38200702 wait complete. 2024-09-12T03:11:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.819+0000 7f663badb740 20 client.25616 trim_cache size 0 max 16384 2024-09-12T03:11:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:11:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:11:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:11:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 msgr2=0x7f66180a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 0x7f66180a5280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6620013600 tx=0x7f662000b530 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 msgr2=0x7f6634198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f662402a9f0 tx=0x7f6624003b90 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 shutdown_connections 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663410bf00 0x7f663419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f66180a2dc0 0x7f66180a5280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6634103400 0x7f66341988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 --2- 172.21.15.57:0/1491094522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6634102ae0 0x7f6634198360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.826+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 >> 172.21.15.57:0/1491094522 conn(0x7f66340777c0 msgr2=0x7f66340fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.827+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 shutdown_connections 2024-09-12T03:11:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:10.827+0000 7f6633200640 1 -- 172.21.15.57:0/1491094522 wait complete. 2024-09-12T03:11:10.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:10 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:11.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:10 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:11.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:10 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:11 smithi181 ceph-mon[35320]: pgmap v160: 129 pgs: 129 active+clean; 35 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 23 KiB/s rd, 240 MiB/s wr, 1.33k op/s 2024-09-12T03:11:12.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:11 smithi057 ceph-mon[30361]: pgmap v160: 129 pgs: 129 active+clean; 35 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 23 KiB/s rd, 240 MiB/s wr, 1.33k op/s 2024-09-12T03:11:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:11 smithi204 ceph-mon[35013]: pgmap v160: 129 pgs: 129 active+clean; 35 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 23 KiB/s rd, 240 MiB/s wr, 1.33k op/s 2024-09-12T03:11:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:13 smithi057 ceph-mon[30361]: pgmap v161: 129 pgs: 129 active+clean; 36 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 30 KiB/s rd, 275 MiB/s wr, 1.69k op/s 2024-09-12T03:11:14.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:13 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:14.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:13 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:14.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:13 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:11:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:13 smithi204 ceph-mon[35013]: pgmap v161: 129 pgs: 129 active+clean; 36 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 30 KiB/s rd, 275 MiB/s wr, 1.69k op/s 2024-09-12T03:11:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:13 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:13 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:14.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:13 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:11:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:13 smithi181 ceph-mon[35320]: pgmap v161: 129 pgs: 129 active+clean; 36 GiB data, 74 GiB used, 998 GiB / 1.0 TiB avail; 30 KiB/s rd, 275 MiB/s wr, 1.69k op/s 2024-09-12T03:11:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:13 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:13 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:13 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:11:16.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:15 smithi057 ceph-mon[30361]: pgmap v162: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 221 MiB/s wr, 1.48k op/s 2024-09-12T03:11:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:15 smithi204 ceph-mon[35013]: pgmap v162: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 221 MiB/s wr, 1.48k op/s 2024-09-12T03:11:16.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:15 smithi181 ceph-mon[35320]: pgmap v162: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 221 MiB/s wr, 1.48k op/s 2024-09-12T03:11:18.123 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:17 smithi057 ceph-mon[30361]: pgmap v163: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 232 MiB/s wr, 1.62k op/s 2024-09-12T03:11:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:17 smithi204 ceph-mon[35013]: pgmap v163: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 232 MiB/s wr, 1.62k op/s 2024-09-12T03:11:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:17 smithi181 ceph-mon[35320]: pgmap v163: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 24 KiB/s rd, 232 MiB/s wr, 1.62k op/s 2024-09-12T03:11:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:19 smithi204 ceph-mon[35013]: pgmap v164: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 997 GiB / 1.0 TiB avail; 21 KiB/s rd, 264 MiB/s wr, 2.04k op/s 2024-09-12T03:11:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:19 smithi181 ceph-mon[35320]: pgmap v164: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 997 GiB / 1.0 TiB avail; 21 KiB/s rd, 264 MiB/s wr, 2.04k op/s 2024-09-12T03:11:20.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:19 smithi057 ceph-mon[30361]: pgmap v164: 129 pgs: 129 active+clean; 36 GiB data, 75 GiB used, 997 GiB / 1.0 TiB avail; 21 KiB/s rd, 264 MiB/s wr, 2.04k op/s 2024-09-12T03:11:21.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:20 smithi204 ceph-mon[35013]: pgmap v165: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 15 KiB/s rd, 221 MiB/s wr, 1.91k op/s 2024-09-12T03:11:21.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:20 smithi181 ceph-mon[35320]: pgmap v165: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 15 KiB/s rd, 221 MiB/s wr, 1.91k op/s 2024-09-12T03:11:21.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:20 smithi057 ceph-mon[30361]: pgmap v165: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 15 KiB/s rd, 221 MiB/s wr, 1.91k op/s 2024-09-12T03:11:23.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:23 smithi057 ceph-mon[30361]: pgmap v166: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 2.27k op/s 2024-09-12T03:11:23.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:23 smithi204 ceph-mon[35013]: pgmap v166: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 2.27k op/s 2024-09-12T03:11:23.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:23 smithi181 ceph-mon[35320]: pgmap v166: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 2.27k op/s 2024-09-12T03:11:25.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:25 smithi057 ceph-mon[30361]: pgmap v167: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 12 KiB/s rd, 187 MiB/s wr, 1.81k op/s 2024-09-12T03:11:25.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:25 smithi204 ceph-mon[35013]: pgmap v167: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 12 KiB/s rd, 187 MiB/s wr, 1.81k op/s 2024-09-12T03:11:25.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:25 smithi181 ceph-mon[35320]: pgmap v167: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 12 KiB/s rd, 187 MiB/s wr, 1.81k op/s 2024-09-12T03:11:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.034+0000 7fdc65e00640 1 -- 172.21.15.57:0/2106462237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60109460 msgr2=0x7fdc60109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.034+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2106462237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60109460 0x7fdc60109840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009900 tx=0x7fdc5402a410 comp rx=0 tx=0).stop 2024-09-12T03:11:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.035+0000 7fdc65e00640 1 -- 172.21.15.57:0/2106462237 shutdown_connections 2024-09-12T03:11:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.035+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2106462237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc60112920 0x7fdc60114db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.035+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2106462237 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109e10 0x7fdc60112260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.035+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2106462237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60109460 0x7fdc60109840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.035+0000 7fdc65e00640 1 -- 172.21.15.57:0/2106462237 >> 172.21.15.57:0/2106462237 conn(0x7fdc6006cbf0 msgr2=0x7fdc6006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.037+0000 7fdc65e00640 1 -- 172.21.15.57:0/2106462237 shutdown_connections 2024-09-12T03:11:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.037+0000 7fdc65e00640 1 -- 172.21.15.57:0/2106462237 wait complete. 2024-09-12T03:11:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.042+0000 7fdc65e00640 1 Processor -- start 2024-09-12T03:11:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.044+0000 7fdc65e00640 1 -- start start 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 0x7fdc60111f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60112920 0x7fdc6010c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60117040 con 0x7fdc60109460 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60116ec0 con 0x7fdc60112920 2024-09-12T03:11:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.045+0000 7fdc65e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601171c0 con 0x7fdc6010c560 2024-09-12T03:11:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.047+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60112920 0x7fdc6010c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.047+0000 7fdc65400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.047+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 0x7fdc60111f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.047+0000 7fdc65400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:45268/0 (socket says 172.21.15.57:45268) 2024-09-12T03:11:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.047+0000 7fdc65400640 1 -- 172.21.15.57:0/2737504441 learned_addr learned my addr 172.21.15.57:0/2737504441 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.048+0000 7fdc65400640 1 -- 172.21.15.57:0/2737504441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60112920 msgr2=0x7fdc6010c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.048+0000 7fdc65400640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60112920 0x7fdc6010c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.048+0000 7fdc65400640 1 -- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 msgr2=0x7fdc60111f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.048+0000 7fdc65400640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 0x7fdc60111f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.048+0000 7fdc65400640 1 -- 172.21.15.57:0/2737504441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601bbef0 con 0x7fdc6010c560 2024-09-12T03:11:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.050+0000 7fdc64a00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 0x7fdc60111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.051+0000 7fdc65400640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800eb70 tx=0x7fdc5800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.051+0000 7fdc5d600640 1 -- 172.21.15.57:0/2737504441 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004d20 con 0x7fdc6010c560 2024-09-12T03:11:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.052+0000 7fdc5d600640 1 -- 172.21.15.57:0/2737504441 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5800ee70 con 0x7fdc6010c560 2024-09-12T03:11:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.052+0000 7fdc5d600640 1 -- 172.21.15.57:0/2737504441 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58010760 con 0x7fdc6010c560 2024-09-12T03:11:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.053+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601bc1e0 con 0x7fdc6010c560 2024-09-12T03:11:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.054+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601bc6e0 con 0x7fdc6010c560 2024-09-12T03:11:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.073+0000 7fdc6a28f740 1 Processor -- start 2024-09-12T03:11:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.076+0000 7fdc5d600640 1 -- 172.21.15.57:0/2737504441 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fdc58004ec0 con 0x7fdc6010c560 2024-09-12T03:11:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.081+0000 7fdc6a28f740 1 -- start start 2024-09-12T03:11:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.081+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 0x56414a472790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.081+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.082+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a47f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.082+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56414a3d8100 con 0x56414a472cd0 2024-09-12T03:11:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.082+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56414a3801b0 con 0x56414a4723b0 2024-09-12T03:11:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.082+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56414a3d40c0 con 0x56414a47d570 2024-09-12T03:11:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.085+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.086+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33042/0 (socket says 172.21.15.57:33042) 2024-09-12T03:11:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.086+0000 7fdc64a00640 1 -- 172.21.15.57:0/2480383345 learned_addr learned my addr 172.21.15.57:0/2480383345 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.088+0000 7fdc5d600640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 0x7fdc480a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.088+0000 7fdc5d600640 1 -- 172.21.15.57:0/2737504441 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fdc580cfec0 con 0x7fdc6010c560 2024-09-12T03:11:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.089+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 0x56414a472790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.089+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a47f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.090+0000 7fdc64a00640 1 -- 172.21.15.57:0/2480383345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 msgr2=0x56414a47f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.090+0000 7fdc64a00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a47f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.090+0000 7fdc64a00640 1 -- 172.21.15.57:0/2480383345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 msgr2=0x56414a472790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.090+0000 7fdc64a00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 0x56414a472790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.090+0000 7fdc64a00640 1 -- 172.21.15.57:0/2480383345 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414a0eb210 con 0x56414a472cd0 2024-09-12T03:11:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.091+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 0x56414a472790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:11:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.095+0000 7fdc65400640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 0x7fdc480a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.099 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.098+0000 7fdc64a00640 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005dc0 tx=0x7fdc54031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46899c29ceb1af72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.102+0000 7fdc51c00640 1 -- 172.21.15.57:0/2480383345 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54035030 con 0x56414a472cd0 2024-09-12T03:11:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.104+0000 7fdc65400640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 0x7fdc480a5300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc580cd260 tx=0x7fdc58009200 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.107+0000 7fdc51c00640 1 -- 172.21.15.57:0/2480383345 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc540043c0 con 0x56414a472cd0 2024-09-12T03:11:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.107+0000 7fdc51c00640 1 -- 172.21.15.57:0/2480383345 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004d90 con 0x56414a472cd0 2024-09-12T03:11:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.109+0000 7fdc6a28f740 1 -- 172.21.15.57:0/2480383345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 msgr2=0x56414a47d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.110+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005dc0 tx=0x7fdc54031510 comp rx=0 tx=0).stop 2024-09-12T03:11:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.116+0000 7fdc6a28f740 1 -- 172.21.15.57:0/2480383345 shutdown_connections 2024-09-12T03:11:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.117+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a47f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.117+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a472cd0 0x56414a47d030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.117+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/2480383345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a4723b0 0x56414a472790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.117+0000 7fdc6a28f740 1 -- 172.21.15.57:0/2480383345 >> 172.21.15.57:0/2480383345 conn(0x56414a46e920 msgr2=0x56414a46ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.118+0000 7fdc6a28f740 1 -- 172.21.15.57:0/2480383345 shutdown_connections 2024-09-12T03:11:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.118+0000 7fdc6a28f740 1 -- 172.21.15.57:0/2480383345 wait complete. 2024-09-12T03:11:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.119+0000 7fdc6a28f740 1 -- start start 2024-09-12T03:11:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.119+0000 7fdc6a28f740 1 Processor -- start 2024-09-12T03:11:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a47aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a47af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56414a4800e0 con 0x56414a4723b0 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56414a361090 con 0x56414a47d570 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.124+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56414a480260 con 0x56414a472cd0 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.125+0000 7fdc6a28f740 10 client.0 _ll_register_callbacks cb 0x56414a3b5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.128+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a47af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.128+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a47aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.128+0000 7fdc64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a47aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:45280/0 (socket says 172.21.15.57:45280) 2024-09-12T03:11:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.128+0000 7fdc64a00640 1 -- 172.21.15.57:0/546020729 learned_addr learned my addr 172.21.15.57:0/546020729 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.128+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.129+0000 7fdc5fe00640 1 -- 172.21.15.57:0/546020729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 msgr2=0x56414a47aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.129+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a47aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.129+0000 7fdc5fe00640 1 -- 172.21.15.57:0/546020729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 msgr2=0x56414a47a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.129+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.129+0000 7fdc5fe00640 1 -- 172.21.15.57:0/546020729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414a1b7620 con 0x56414a47d570 2024-09-12T03:11:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.130+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a47af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009fd0 tx=0x7fdc4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85309b145f96c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.130+0000 7fdc46000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:11:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.132+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c01a040 con 0x56414a47d570 2024-09-12T03:11:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.133+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fdc4c011270 con 0x56414a47d570 2024-09-12T03:11:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.134+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc540043c0 con 0x56414a47d570 2024-09-12T03:11:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.134+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c004020 con 0x56414a47d570 2024-09-12T03:11:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.135+0000 7fdc6a28f740 10 client.25628 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.135+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56414a48bea0 con 0x56414a47d570 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc540043c0 con 0x56414a47d570 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdc4c003ca0 con 0x56414a47d570 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc6a28f740 10 client.25628 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc6a28f740 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc6a28f740 20 client.25628 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.136+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a48d310 0x56414a48f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.137+0000 7fdc6a28f740 4 client.25628 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.137+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414a3e5830 con 0x56414a48d310 2024-09-12T03:11:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.137+0000 7fdc64a00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a48d310 0x56414a48f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.139+0000 7fdc64a00640 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a48d310 0x56414a48f700 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f5ee2947f61997 server_cookie=cc74fb00764531db in_seq=0 out_seq=0 2024-09-12T03:11:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc46000640 10 client.25628 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc46000640 1 -- 172.21.15.57:0/546020729 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56414a3e5830 con 0x56414a48d310 2024-09-12T03:11:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc46000640 10 client.25628 handle_command_reply: tid=0 2024-09-12T03:11:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 client.25628 shutdown 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 msgr2=0x56414a47af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a47af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009fd0 tx=0x7fdc4c004730 comp rx=0 tx=0).stop 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 shutdown_connections 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a48d310 0x56414a48f700 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a47af80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a47aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/546020729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 >> 172.21.15.57:0/546020729 conn(0x56414a46e920 msgr2=0x56414a474680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.140+0000 7fdc46000640 0 client.25628 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.141+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 shutdown_connections 2024-09-12T03:11:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.141+0000 7fdc6a28f740 1 -- 172.21.15.57:0/546020729 wait complete. 2024-09-12T03:11:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.142+0000 7fdc6a28f740 20 client.25628 trim_cache size 0 max 16384 2024-09-12T03:11:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.183+0000 7fdc6a28f740 1 Processor -- start 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 -- start start 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a46d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c8e10 con 0x56414a4723b0 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c66e0 con 0x56414a47d570 2024-09-12T03:11:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.184+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c4530 con 0x56414a472cd0 2024-09-12T03:11:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.185+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.185+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:36642/0 (socket says 172.21.15.57:36642) 2024-09-12T03:11:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.185+0000 7fdc5fe00640 1 -- 172.21.15.57:0/3423939254 learned_addr learned my addr 172.21.15.57:0/3423939254 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.186+0000 7fdc64a00640 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a46d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.187+0000 7fdc64a00640 1 -- 172.21.15.57:0/3423939254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 msgr2=0x56414a495010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.187+0000 7fdc64a00640 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.187+0000 7fdc64a00640 1 -- 172.21.15.57:0/3423939254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 msgr2=0x56414a47adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.187+0000 7fdc64a00640 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.187+0000 7fdc64a00640 1 -- 172.21.15.57:0/3423939254 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414a46e750 con 0x56414a472cd0 2024-09-12T03:11:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.188+0000 7fdc64a00640 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a46d8d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x56414a479d70 tx=0x7fdc54005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed00777fbf4c8e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.188+0000 7fdc51200640 1 -- 172.21.15.57:0/3423939254 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54035040 con 0x56414a472cd0 2024-09-12T03:11:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.188+0000 7fdc51200640 1 -- 172.21.15.57:0/3423939254 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc540055a0 con 0x56414a472cd0 2024-09-12T03:11:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.188+0000 7fdc51200640 1 -- 172.21.15.57:0/3423939254 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540035f0 con 0x56414a472cd0 2024-09-12T03:11:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.188+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-12T03:11:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.190+0000 7fdc6a28f740 1 -- 172.21.15.57:0/3423939254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 msgr2=0x56414a46d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.190+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a46d8d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x56414a479d70 tx=0x7fdc54005d50 comp rx=0 tx=0).stop 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.199+0000 7fdc6a28f740 1 -- 172.21.15.57:0/3423939254 shutdown_connections 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.199+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a47d570 0x56414a495010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.199+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a472cd0 0x56414a46d8d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.199+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/3423939254 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a47adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.199+0000 7fdc6a28f740 1 -- 172.21.15.57:0/3423939254 >> 172.21.15.57:0/3423939254 conn(0x56414a46e920 msgr2=0x56414a46d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.200+0000 7fdc6a28f740 1 -- 172.21.15.57:0/3423939254 shutdown_connections 2024-09-12T03:11:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.200+0000 7fdc6a28f740 1 -- 172.21.15.57:0/3423939254 wait complete. 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.200+0000 7fdc6a28f740 1 -- start start 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 Processor -- start 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a472cd0 0x56414a481220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a4effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c4ba0 con 0x56414a4723b0 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c6010 con 0x56414a472cd0 2024-09-12T03:11:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601c5270 con 0x56414a47d570 2024-09-12T03:11:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.201+0000 7fdc6a28f740 10 client.0 _ll_register_callbacks cb 0x7fdc4c012560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.202+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.202+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:33058/0 (socket says 172.21.15.57:33058) 2024-09-12T03:11:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.202+0000 7fdc5fe00640 1 -- 172.21.15.57:0/1203071796 learned_addr learned my addr 172.21.15.57:0/1203071796 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.202+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a4effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.203+0000 7fdc5fe00640 1 -- 172.21.15.57:0/1203071796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 msgr2=0x56414a4effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.203+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a4effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.203+0000 7fdc5fe00640 1 -- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a472cd0 msgr2=0x56414a481220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.203+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a472cd0 0x56414a481220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.203+0000 7fdc5fe00640 1 -- 172.21.15.57:0/1203071796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414a4f0640 con 0x56414a4723b0 2024-09-12T03:11:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.207+0000 7fdc5fe00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c012680 tx=0x7fdc4c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c12dd6504fcfac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.208+0000 7fdc46000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:11:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.208+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c021c70 con 0x56414a4723b0 2024-09-12T03:11:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.208+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7fdc4c005230 con 0x56414a4723b0 2024-09-12T03:11:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.208+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc540055a0 con 0x56414a4723b0 2024-09-12T03:11:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.209+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c005640 con 0x56414a4723b0 2024-09-12T03:11:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.209+0000 7fdc6a28f740 10 client.15501 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.209+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56414a4f79b0 con 0x56414a4723b0 2024-09-12T03:11:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.209+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4c003ad0 con 0x56414a4723b0 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdc4c02d830 con 0x56414a4723b0 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 20 client.15501 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a4f8060 0x56414a4fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 4 client.15501 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:11:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.210+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdc540055a0 con 0x56414a4f8060 2024-09-12T03:11:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.211+0000 7fdc64a00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a4f8060 0x56414a4fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.211+0000 7fdc64a00640 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a4f8060 0x56414a4fa480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78da1c140b6c1dfd server_cookie=33c1945304de1ea in_seq=0 out_seq=0 2024-09-12T03:11:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.211+0000 7fdc46000640 10 client.15501 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.463+0000 7fdc46000640 1 -- 172.21.15.57:0/1203071796 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdc540055a0 con 0x56414a4f8060 2024-09-12T03:11:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.463+0000 7fdc46000640 10 client.15501 handle_command_reply: tid=0 2024-09-12T03:11:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.463+0000 7fdc6a28f740 1 client.15501 shutdown 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 msgr2=0x56414a480ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c012680 tx=0x7fdc4c009310 comp rx=0 tx=0).stop 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 shutdown_connections 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56414a4f8060 0x56414a4fa480 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56414a47d570 0x56414a4effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56414a472cd0 0x56414a481220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 --2- 172.21.15.57:0/1203071796 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56414a4723b0 0x56414a480ce0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 >> 172.21.15.57:0/1203071796 conn(0x56414a46e920 msgr2=0x56414a46d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.464+0000 7fdc46000640 0 client.15501 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.465+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 shutdown_connections 2024-09-12T03:11:26.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.465+0000 7fdc6a28f740 1 -- 172.21.15.57:0/1203071796 wait complete. 2024-09-12T03:11:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.465+0000 7fdc6a28f740 20 client.15501 trim_cache size 0 max 16384 2024-09-12T03:11:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:11:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:11:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:11:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 msgr2=0x7fdc480a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 0x7fdc480a5300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc580cd260 tx=0x7fdc58009200 comp rx=0 tx=0).stop 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 msgr2=0x7fdc6010ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800eb70 tx=0x7fdc5800c6a0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 shutdown_connections 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc6010c560 0x7fdc6010ca10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc60112920 0x7fdc6010c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.469+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdc480a2e40 0x7fdc480a5300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.470+0000 7fdc65e00640 1 --2- 172.21.15.57:0/2737504441 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdc60109460 0x7fdc60111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.470+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 >> 172.21.15.57:0/2737504441 conn(0x7fdc6006cbf0 msgr2=0x7fdc60070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.480+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 shutdown_connections 2024-09-12T03:11:26.479 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:26.480+0000 7fdc65e00640 1 -- 172.21.15.57:0/2737504441 wait complete. 2024-09-12T03:11:27.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:27 smithi057 ceph-mon[30361]: pgmap v168: 129 pgs: 129 active+clean; 37 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 13 KiB/s rd, 196 MiB/s wr, 1.93k op/s 2024-09-12T03:11:27.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:27 smithi204 ceph-mon[35013]: pgmap v168: 129 pgs: 129 active+clean; 37 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 13 KiB/s rd, 196 MiB/s wr, 1.93k op/s 2024-09-12T03:11:27.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:27 smithi181 ceph-mon[35320]: pgmap v168: 129 pgs: 129 active+clean; 37 GiB data, 77 GiB used, 996 GiB / 1.0 TiB avail; 13 KiB/s rd, 196 MiB/s wr, 1.93k op/s 2024-09-12T03:11:29.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:29 smithi057 ceph-mon[30361]: pgmap v169: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 2.22k op/s 2024-09-12T03:11:29.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:29 smithi204 ceph-mon[35013]: pgmap v169: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 2.22k op/s 2024-09-12T03:11:29.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:29 smithi181 ceph-mon[35320]: pgmap v169: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 17 KiB/s rd, 220 MiB/s wr, 2.22k op/s 2024-09-12T03:11:31.624 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:31 smithi057 ceph-mon[30361]: pgmap v170: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 11 KiB/s rd, 172 MiB/s wr, 1.74k op/s 2024-09-12T03:11:31.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:31 smithi204 ceph-mon[35013]: pgmap v170: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 11 KiB/s rd, 172 MiB/s wr, 1.74k op/s 2024-09-12T03:11:31.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:31 smithi181 ceph-mon[35320]: pgmap v170: 129 pgs: 129 active+clean; 37 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 11 KiB/s rd, 172 MiB/s wr, 1.74k op/s 2024-09-12T03:11:33.623 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:33 smithi057 ceph-mon[30361]: pgmap v171: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 85 KiB/s rd, 217 MiB/s wr, 2.05k op/s 2024-09-12T03:11:33.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:33 smithi204 ceph-mon[35013]: pgmap v171: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 85 KiB/s rd, 217 MiB/s wr, 2.05k op/s 2024-09-12T03:11:33.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:33 smithi181 ceph-mon[35320]: pgmap v171: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 85 KiB/s rd, 217 MiB/s wr, 2.05k op/s 2024-09-12T03:11:35.628 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:35 smithi057 ceph-mon[30361]: pgmap v172: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 79 KiB/s rd, 159 MiB/s wr, 1.50k op/s 2024-09-12T03:11:35.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:35 smithi204 ceph-mon[35013]: pgmap v172: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 79 KiB/s rd, 159 MiB/s wr, 1.50k op/s 2024-09-12T03:11:35.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:35 smithi181 ceph-mon[35320]: pgmap v172: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 993 GiB / 1.0 TiB avail; 79 KiB/s rd, 159 MiB/s wr, 1.50k op/s 2024-09-12T03:11:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:37 smithi057 ceph-mon[30361]: pgmap v173: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 79 KiB/s rd, 188 MiB/s wr, 1.66k op/s 2024-09-12T03:11:37.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:37 smithi204 ceph-mon[35013]: pgmap v173: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 79 KiB/s rd, 188 MiB/s wr, 1.66k op/s 2024-09-12T03:11:37.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:37 smithi181 ceph-mon[35320]: pgmap v173: 129 pgs: 129 active+clean; 38 GiB data, 80 GiB used, 992 GiB / 1.0 TiB avail; 79 KiB/s rd, 188 MiB/s wr, 1.66k op/s 2024-09-12T03:11:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:39 smithi057 ceph-mon[30361]: pgmap v174: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 86 KiB/s rd, 227 MiB/s wr, 1.84k op/s 2024-09-12T03:11:39.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:39 smithi204 ceph-mon[35013]: pgmap v174: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 86 KiB/s rd, 227 MiB/s wr, 1.84k op/s 2024-09-12T03:11:39.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:39 smithi181 ceph-mon[35320]: pgmap v174: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 86 KiB/s rd, 227 MiB/s wr, 1.84k op/s 2024-09-12T03:11:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.903+0000 7f8d98a00640 1 -- 172.21.15.57:0/3010267228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d9410df80 msgr2=0x7f8d9410e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.903+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3010267228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d9410df80 0x7f8d9410e360 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009890 tx=0x7f8d7c02a370 comp rx=0 tx=0).stop 2024-09-12T03:11:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 -- 172.21.15.57:0/3010267228 shutdown_connections 2024-09-12T03:11:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3010267228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d941089a0 0x7f8d94114d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3010267228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d94108020 0x7f8d94108460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.906 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3010267228 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d9410df80 0x7f8d9410e360 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 -- 172.21.15.57:0/3010267228 >> 172.21.15.57:0/3010267228 conn(0x7f8d94077860 msgr2=0x7f8d94079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 -- 172.21.15.57:0/3010267228 shutdown_connections 2024-09-12T03:11:39.907 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.906+0000 7f8d98a00640 1 -- 172.21.15.57:0/3010267228 wait complete. 2024-09-12T03:11:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.908+0000 7f8d98a00640 1 Processor -- start 2024-09-12T03:11:39.908 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.908+0000 7f8d98a00640 1 -- start start 2024-09-12T03:11:39.911 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.911+0000 7f8d98a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d94108020 0x7f8d9407af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.912 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.911+0000 7f8d98a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d941089a0 0x7f8d9407b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d98a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d98a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941182b0 con 0x7f8d94108020 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d98a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94118130 con 0x7f8d941089a0 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d98a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94118430 con 0x7f8d9410df80 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60692/0 (socket says 172.21.15.57:60692) 2024-09-12T03:11:39.913 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.912+0000 7f8d93e00640 1 -- 172.21.15.57:0/3042667136 learned_addr learned my addr 172.21.15.57:0/3042667136 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.913+0000 7f8d93e00640 1 -- 172.21.15.57:0/3042667136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d941089a0 msgr2=0x7f8d9407b4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:11:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.913+0000 7f8d93e00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d941089a0 0x7f8d9407b4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.913+0000 7f8d93e00640 1 -- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d94108020 msgr2=0x7f8d9407af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.913+0000 7f8d93e00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d94108020 0x7f8d9407af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.915 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.913+0000 7f8d93e00640 1 -- 172.21.15.57:0/3042667136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d941b3230 con 0x7f8d9410df80 2024-09-12T03:11:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.915+0000 7f8d93e00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400d750 tx=0x7f8d8400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.915+0000 7f8d8be00640 1 -- 172.21.15.57:0/3042667136 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84017070 con 0x7f8d9410df80 2024-09-12T03:11:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.916+0000 7f8d8be00640 1 -- 172.21.15.57:0/3042667136 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d84005500 con 0x7f8d9410df80 2024-09-12T03:11:39.916 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.916+0000 7f8d8be00640 1 -- 172.21.15.57:0/3042667136 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84004d90 con 0x7f8d9410df80 2024-09-12T03:11:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.916+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d941b3520 con 0x7f8d9410df80 2024-09-12T03:11:39.917 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.916+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d941b3a20 con 0x7f8d9410df80 2024-09-12T03:11:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.928+0000 7f8d9ce7c740 1 Processor -- start 2024-09-12T03:11:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.931+0000 7f8d8be00640 1 -- 172.21.15.57:0/3042667136 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f8d84004f30 con 0x7f8d9410df80 2024-09-12T03:11:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 -- start start 2024-09-12T03:11:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e411932100 con 0x55e4119cc3b0 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e4118da1b0 con 0x55e4119cccd0 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.932+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e41192e0c0 con 0x55e4119d7570 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:56684/0 (socket says 172.21.15.57:56684) 2024-09-12T03:11:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 -- 172.21.15.57:0/2801215135 learned_addr learned my addr 172.21.15.57:0/2801215135 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d92a00640 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 -- 172.21.15.57:0/2801215135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 msgr2=0x55e4119d9960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d9960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 -- 172.21.15.57:0/2801215135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 msgr2=0x55e4119cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.933+0000 7f8d93400640 1 -- 172.21.15.57:0/2801215135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e411645210 con 0x55e4119cccd0 2024-09-12T03:11:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.934+0000 7f8d92a00640 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.934+0000 7f8d8be00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 0x7f8d780a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.934+0000 7f8d93400640 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009860 tx=0x7f8d7c009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61da9da727dfb3e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.934+0000 7f8d8be00640 1 -- 172.21.15.57:0/3042667136 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f8d840cf150 con 0x7f8d9410df80 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.935+0000 7f8d92a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 0x7f8d780a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.935+0000 7f8d83e00640 1 -- 172.21.15.57:0/2801215135 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c037040 con 0x55e4119cccd0 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.936+0000 7f8d92a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 0x7f8d780a5300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55e4119c9ca0 tx=0x7f8d8c0095c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.936+0000 7f8d83e00640 1 -- 172.21.15.57:0/2801215135 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c030340 con 0x55e4119cccd0 2024-09-12T03:11:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.936+0000 7f8d83e00640 1 -- 172.21.15.57:0/2801215135 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c030940 con 0x55e4119cccd0 2024-09-12T03:11:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.937+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2801215135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 msgr2=0x55e4119d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.937+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009860 tx=0x7f8d7c009490 comp rx=0 tx=0).stop 2024-09-12T03:11:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2801215135 shutdown_connections 2024-09-12T03:11:39.938 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d7030 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2801215135 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2801215135 >> 172.21.15.57:0/2801215135 conn(0x55e4119c8920 msgr2=0x55e4119c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2801215135 shutdown_connections 2024-09-12T03:11:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.938+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2801215135 wait complete. 2024-09-12T03:11:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.939+0000 7f8d9ce7c740 1 -- start start 2024-09-12T03:11:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.939+0000 7f8d9ce7c740 1 Processor -- start 2024-09-12T03:11:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e4119da020 con 0x55e4119cc3b0 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e4118bb090 con 0x55e4119cccd0 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4119da1a0 con 0x55e4119d7570 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d9ce7c740 10 client.0 _ll_register_callbacks cb 0x55e41190ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.940+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.941 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57352/0 (socket says 172.21.15.57:57352) 2024-09-12T03:11:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 -- 172.21.15.57:0/913852053 learned_addr learned my addr 172.21.15.57:0/913852053 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93e00640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d92a00640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 -- 172.21.15.57:0/913852053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 msgr2=0x55e4119d4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 -- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 msgr2=0x55e4119d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.941+0000 7f8d93400640 1 -- 172.21.15.57:0/913852053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e411711620 con 0x55e4119cc3b0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.942+0000 7f8d93400640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005940 tx=0x7f8d7c005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73936491d7c8af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.942+0000 7f8d80c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.942+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c049060 con 0x55e4119cc3b0 2024-09-12T03:11:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.942+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c030340 con 0x55e4119cc3b0 2024-09-12T03:11:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.942+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f8d7c030890 con 0x55e4119cc3b0 2024-09-12T03:11:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d7c050050 con 0x55e4119cc3b0 2024-09-12T03:11:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d9ce7c740 10 client.15507 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4119e5d10 con 0x55e4119cc3b0 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c030340 con 0x55e4119cc3b0 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d7c03a070 con 0x55e4119cc3b0 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d9ce7c740 10 client.15507 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.943+0000 7f8d9ce7c740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.944+0000 7f8d9ce7c740 20 client.15507 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.944+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e4119e7060 0x55e4119e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.944+0000 7f8d9ce7c740 4 client.15507 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.944+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e41193f830 con 0x55e4119e7060 2024-09-12T03:11:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.944+0000 7f8d92a00640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e4119e7060 0x55e4119e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.945+0000 7f8d92a00640 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e4119e7060 0x55e4119e9450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3a9b9e7296773e server_cookie=820c5523e72417b3 in_seq=0 out_seq=0 2024-09-12T03:11:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.945+0000 7f8d80c00640 10 client.15507 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d80c00640 1 -- 172.21.15.57:0/913852053 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e41193f830 con 0x55e4119e7060 2024-09-12T03:11:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d80c00640 10 client.15507 handle_command_reply: tid=0 2024-09-12T03:11:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 client.15507 shutdown 2024-09-12T03:11:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 msgr2=0x55e4119d4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005940 tx=0x7f8d7c005190 comp rx=0 tx=0).stop 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 shutdown_connections 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e4119e7060 0x55e4119e9450 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e4119d4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/913852053 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d4490 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 >> 172.21.15.57:0/913852053 conn(0x55e4119c8920 msgr2=0x55e4119ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.946+0000 7f8d80c00640 0 client.15507 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.947+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 shutdown_connections 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.947+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/913852053 wait complete. 2024-09-12T03:11:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.947+0000 7f8d9ce7c740 20 client.15507 trim_cache size 0 max 16384 2024-09-12T03:11:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 Processor -- start 2024-09-12T03:11:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 -- start start 2024-09-12T03:11:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119cccd0 0x55e4119eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 0x55e4119eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941c0560 con 0x55e4119cc3b0 2024-09-12T03:11:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941bde30 con 0x55e4119d7570 2024-09-12T03:11:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.966+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941bbc80 con 0x55e4119cccd0 2024-09-12T03:11:39.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.970+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.970+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57354/0 (socket says 172.21.15.57:57354) 2024-09-12T03:11:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.970+0000 7f8d93400640 1 -- 172.21.15.57:0/2507468428 learned_addr learned my addr 172.21.15.57:0/2507468428 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.970+0000 7f8d92a00640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119cccd0 0x55e4119eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.970+0000 7f8d93e00640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 0x55e4119eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93e00640 1 -- 172.21.15.57:0/2507468428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119cccd0 msgr2=0x55e4119eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93e00640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119cccd0 0x55e4119eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93e00640 1 -- 172.21.15.57:0/2507468428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 msgr2=0x55e4119d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93e00640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93e00640 1 -- 172.21.15.57:0/2507468428 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4119c8750 con 0x55e4119d7570 2024-09-12T03:11:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.971+0000 7f8d93400640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.972+0000 7f8d93e00640 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 0x55e4119eeff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d840cc4f0 tx=0x7f8d84097690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94294326714c8026 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.973+0000 7f8d83400640 1 -- 172.21.15.57:0/2507468428 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84020040 con 0x55e4119d7570 2024-09-12T03:11:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.973+0000 7f8d83400640 1 -- 172.21.15.57:0/2507468428 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8409b9c0 con 0x55e4119d7570 2024-09-12T03:11:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.973+0000 7f8d83400640 1 -- 172.21.15.57:0/2507468428 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84097930 con 0x55e4119d7570 2024-09-12T03:11:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.975+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2507468428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 msgr2=0x55e4119eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.975+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 0x55e4119eeff0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d840cc4f0 tx=0x7f8d84097690 comp rx=0 tx=0).stop 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.976+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2507468428 shutdown_connections 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.976+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119d7570 0x55e4119eeff0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.976+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119cccd0 0x55e4119eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.976+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/2507468428 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.976+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2507468428 >> 172.21.15.57:0/2507468428 conn(0x55e4119c8920 msgr2=0x55e4119d5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.978+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2507468428 shutdown_connections 2024-09-12T03:11:39.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.978+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/2507468428 wait complete. 2024-09-12T03:11:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.979+0000 7f8d9ce7c740 1 -- start start 2024-09-12T03:11:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.979+0000 7f8d9ce7c740 1 Processor -- start 2024-09-12T03:11:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.980+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e411a4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941bc2f0 con 0x55e4119cc3b0 2024-09-12T03:11:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941bd760 con 0x55e4119cccd0 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941bc9c0 con 0x55e4119d7570 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.981+0000 7f8d9ce7c740 10 client.0 _ll_register_callbacks cb 0x7f8d8c009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:57358/0 (socket says 172.21.15.57:57358) 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93400640 1 -- 172.21.15.57:0/1834678963 learned_addr learned my addr 172.21.15.57:0/1834678963 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d92a00640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e411a4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 -- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 msgr2=0x55e4119dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119dada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 -- 172.21.15.57:0/1834678963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 msgr2=0x55e4119da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93e00640 1 -- 172.21.15.57:0/1834678963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e411a4ffb0 con 0x55e4119d7570 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.982+0000 7f8d93400640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d93e00640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e411a4fa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d840a40f0 tx=0x7f8d8409f6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6edb32ab364be062 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d80c00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840b9070 con 0x55e4119d7570 2024-09-12T03:11:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f8d84005220 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8409b9c0 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d840a3f30 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d9ce7c740 10 client.25666 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.983+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e411a511c0 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8409b9c0 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d840bb240 con 0x55e4119d7570 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 10 client.25666 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 20 client.25666 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e411a51880 0x55e411a53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 4 client.25666 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e411939b80 con 0x55e411a51880 2024-09-12T03:11:39.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.984+0000 7f8d93400640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e411a51880 0x55e411a53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.985+0000 7f8d93400640 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e411a51880 0x55e411a53c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e77c679d93090a4 server_cookie=e6cf3f24c53bacf0 in_seq=0 out_seq=0 2024-09-12T03:11:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.985+0000 7f8d80c00640 10 client.25666 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d80c00640 1 -- 172.21.15.57:0/1834678963 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e411939b80 con 0x55e411a51880 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d80c00640 10 client.25666 handle_command_reply: tid=0 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 client.25666 shutdown 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 msgr2=0x55e411a4fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e411a4fa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d840a40f0 tx=0x7f8d8409f6f0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 shutdown_connections 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e411a51880 0x55e411a53c70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e4119d7570 0x55e411a4fa70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d80c00640 0 client.25666 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e4119cccd0 0x55e4119dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 --2- 172.21.15.57:0/1834678963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e4119cc3b0 0x55e4119da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.993+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 >> 172.21.15.57:0/1834678963 conn(0x55e4119c8920 msgr2=0x55e4119c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.994+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 shutdown_connections 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.994+0000 7f8d9ce7c740 1 -- 172.21.15.57:0/1834678963 wait complete. 2024-09-12T03:11:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.994+0000 7f8d9ce7c740 20 client.25666 trim_cache size 0 max 16384 2024-09-12T03:11:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:11:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:11:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:11:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:11:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 msgr2=0x7f8d780a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 0x7f8d780a5300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55e4119c9ca0 tx=0x7f8d8c0095c0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 msgr2=0x7f8d9407ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400d750 tx=0x7f8d8400dc10 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 shutdown_connections 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d9410df80 0x7f8d9407ba10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d941089a0 0x7f8d9407b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8d780a2e40 0x7f8d780a5300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 --2- 172.21.15.57:0/3042667136 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8d94108020 0x7f8d9407af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 >> 172.21.15.57:0/3042667136 conn(0x7f8d94077860 msgr2=0x7f8d94078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 shutdown_connections 2024-09-12T03:11:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:39.995+0000 7f8d98a00640 1 -- 172.21.15.57:0/3042667136 wait complete. 2024-09-12T03:11:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:41 smithi057 ceph-mon[30361]: pgmap v175: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 81 KiB/s rd, 194 MiB/s wr, 1.43k op/s 2024-09-12T03:11:41.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:41 smithi204 ceph-mon[35013]: pgmap v175: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 81 KiB/s rd, 194 MiB/s wr, 1.43k op/s 2024-09-12T03:11:41.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:41 smithi181 ceph-mon[35320]: pgmap v175: 129 pgs: 129 active+clean; 39 GiB data, 82 GiB used, 991 GiB / 1.0 TiB avail; 81 KiB/s rd, 194 MiB/s wr, 1.43k op/s 2024-09-12T03:11:43.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:43 smithi204 ceph-mon[35013]: pgmap v176: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 88 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-12T03:11:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:43 smithi181 ceph-mon[35320]: pgmap v176: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 88 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-12T03:11:43.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:43 smithi057 ceph-mon[30361]: pgmap v176: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 88 KiB/s rd, 268 MiB/s wr, 1.90k op/s 2024-09-12T03:11:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:45 smithi204 ceph-mon[35013]: pgmap v177: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 14 KiB/s rd, 193 MiB/s wr, 1.25k op/s 2024-09-12T03:11:45.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:45 smithi181 ceph-mon[35320]: pgmap v177: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 14 KiB/s rd, 193 MiB/s wr, 1.25k op/s 2024-09-12T03:11:45.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:45 smithi057 ceph-mon[30361]: pgmap v177: 129 pgs: 129 active+clean; 40 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 14 KiB/s rd, 193 MiB/s wr, 1.25k op/s 2024-09-12T03:11:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:47 smithi181 ceph-mon[35320]: pgmap v178: 129 pgs: 129 active+clean; 40 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.47k op/s 2024-09-12T03:11:47.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:47 smithi057 ceph-mon[30361]: pgmap v178: 129 pgs: 129 active+clean; 40 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.47k op/s 2024-09-12T03:11:47.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:47 smithi204 ceph-mon[35013]: pgmap v178: 129 pgs: 129 active+clean; 40 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 15 KiB/s rd, 224 MiB/s wr, 1.47k op/s 2024-09-12T03:11:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:49 smithi181 ceph-mon[35320]: pgmap v179: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 21 KiB/s rd, 254 MiB/s wr, 1.67k op/s 2024-09-12T03:11:49.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:49 smithi057 ceph-mon[30361]: pgmap v179: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 21 KiB/s rd, 254 MiB/s wr, 1.67k op/s 2024-09-12T03:11:49.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:49 smithi204 ceph-mon[35013]: pgmap v179: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 21 KiB/s rd, 254 MiB/s wr, 1.67k op/s 2024-09-12T03:11:51.695 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:51 smithi204 ceph-mon[35013]: pgmap v180: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 203 MiB/s wr, 1.34k op/s 2024-09-12T03:11:51.702 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:51 smithi181 ceph-mon[35320]: pgmap v180: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 203 MiB/s wr, 1.34k op/s 2024-09-12T03:11:51.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:51 smithi057 ceph-mon[30361]: pgmap v180: 129 pgs: 129 active+clean; 41 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 203 MiB/s wr, 1.34k op/s 2024-09-12T03:11:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 -- 172.21.15.57:0/402171092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24104910 msgr2=0x7f6c24108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 --2- 172.21.15.57:0/402171092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24104910 0x7f6c24108950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009900 tx=0x7f6c1002a3f0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 -- 172.21.15.57:0/402171092 shutdown_connections 2024-09-12T03:11:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 --2- 172.21.15.57:0/402171092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c241090b0 0x7f6c24110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 --2- 172.21.15.57:0/402171092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24104910 0x7f6c24108950 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 --2- 172.21.15.57:0/402171092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c24103f60 0x7f6c24104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.270+0000 7f6c2b200640 1 -- 172.21.15.57:0/402171092 >> 172.21.15.57:0/402171092 conn(0x7f6c240ff9b0 msgr2=0x7f6c24101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.271+0000 7f6c2b200640 1 -- 172.21.15.57:0/402171092 shutdown_connections 2024-09-12T03:11:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.271+0000 7f6c2b200640 1 -- 172.21.15.57:0/402171092 wait complete. 2024-09-12T03:11:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.272+0000 7f6c2b200640 1 Processor -- start 2024-09-12T03:11:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 -- start start 2024-09-12T03:11:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c241090b0 0x7f6c2410db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 0x7f6c2410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24114180 con 0x7f6c241090b0 2024-09-12T03:11:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24114000 con 0x7f6c24103f60 2024-09-12T03:11:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.273+0000 7f6c2b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24114300 con 0x7f6c2410e0c0 2024-09-12T03:11:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.274+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.274+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48676/0 (socket says 172.21.15.57:48676) 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.274+0000 7f6c23e00640 1 -- 172.21.15.57:0/2226892671 learned_addr learned my addr 172.21.15.57:0/2226892671 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.274+0000 7f6c23400640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c241090b0 0x7f6c2410db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.274+0000 7f6c29200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 0x7f6c2410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 -- 172.21.15.57:0/2226892671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 msgr2=0x7f6c2410e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 0x7f6c2410e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 -- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c241090b0 msgr2=0x7f6c2410db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c241090b0 0x7f6c2410db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 -- 172.21.15.57:0/2226892671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c241ab100 con 0x7f6c24103f60 2024-09-12T03:11:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c29200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 0x7f6c2410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:11:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.275+0000 7f6c23e00640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800c890 tx=0x7f6c1800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.278+0000 7f6c20c00640 1 -- 172.21.15.57:0/2226892671 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18004220 con 0x7f6c24103f60 2024-09-12T03:11:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.279+0000 7f6c20c00640 1 -- 172.21.15.57:0/2226892671 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c180043c0 con 0x7f6c24103f60 2024-09-12T03:11:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.279+0000 7f6c20c00640 1 -- 172.21.15.57:0/2226892671 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180049c0 con 0x7f6c24103f60 2024-09-12T03:11:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.279+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c241ab3f0 con 0x7f6c24103f60 2024-09-12T03:11:53.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.279+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241ab8c0 con 0x7f6c24103f60 2024-09-12T03:11:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.286+0000 7f6c20c00640 1 -- 172.21.15.57:0/2226892671 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f6c18004b60 con 0x7f6c24103f60 2024-09-12T03:11:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.286+0000 7f6c2dffa740 1 Processor -- start 2024-09-12T03:11:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c20c00640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 0x7f6c0c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 -- start start 2024-09-12T03:11:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c20c00640 1 -- 172.21.15.57:0/2226892671 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f6c180cf730 con 0x7f6c24103f60 2024-09-12T03:11:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 0x55c57e910790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c57e869100 con 0x55c57e903b00 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c29200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 0x7f6c0c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c57e8111b0 con 0x55c57e90e3a0 2024-09-12T03:11:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.289+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c57e8650c0 con 0x55c57e9031e0 2024-09-12T03:11:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:48634/0 (socket says 172.21.15.57:48634) 2024-09-12T03:11:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c29200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 0x55c57e910790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c23e00640 1 -- 172.21.15.57:0/11416623 learned_addr learned my addr 172.21.15.57:0/11416623 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c29200640 1 -- 172.21.15.57:0/11416623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 msgr2=0x55c57e9035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c29200640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.290+0000 7f6c29200640 1 -- 172.21.15.57:0/11416623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 msgr2=0x55c57e90de60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c23400640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c29200640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c29200640 1 -- 172.21.15.57:0/11416623 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e57c210 con 0x55c57e90e3a0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c23400640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c23e00640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c29200640 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 0x55c57e910790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c017690 tx=0x7f6c1c017b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be811f3f246c376 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.291+0000 7f6c29200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 0x7f6c0c0a53c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0155c0 tx=0x7f6c1c00f000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.292+0000 7f6c15600640 1 -- 172.21.15.57:0/11416623 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c014040 con 0x55c57e90e3a0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.292+0000 7f6c15600640 1 -- 172.21.15.57:0/11416623 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c009330 con 0x55c57e90e3a0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.292+0000 7f6c15600640 1 -- 172.21.15.57:0/11416623 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01a650 con 0x55c57e90e3a0 2024-09-12T03:11:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 -- 172.21.15.57:0/11416623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 msgr2=0x55c57e910790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 0x55c57e910790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c017690 tx=0x7f6c1c017b50 comp rx=0 tx=0).stop 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 -- 172.21.15.57:0/11416623 shutdown_connections 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90e3a0 0x55c57e910790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/11416623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e9031e0 0x55c57e9035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 -- 172.21.15.57:0/11416623 >> 172.21.15.57:0/11416623 conn(0x55c57e8ff750 msgr2=0x55c57e8ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.293+0000 7f6c2dffa740 1 -- 172.21.15.57:0/11416623 shutdown_connections 2024-09-12T03:11:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.294+0000 7f6c2dffa740 1 -- 172.21.15.57:0/11416623 wait complete. 2024-09-12T03:11:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.294+0000 7f6c2dffa740 1 -- start start 2024-09-12T03:11:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.295+0000 7f6c2dffa740 1 Processor -- start 2024-09-12T03:11:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e90b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c57e910f70 con 0x55c57e903b00 2024-09-12T03:11:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c57e7f2090 con 0x55c57e9031e0 2024-09-12T03:11:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c57e9110f0 con 0x55c57e90e3a0 2024-09-12T03:11:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.297+0000 7f6c2dffa740 10 client.0 _ll_register_callbacks cb 0x55c57e846f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.300+0000 7f6c23400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.300+0000 7f6c23400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50926/0 (socket says 172.21.15.57:50926) 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.300+0000 7f6c23400640 1 -- 172.21.15.57:0/595765736 learned_addr learned my addr 172.21.15.57:0/595765736 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.301+0000 7f6c23400640 1 -- 172.21.15.57:0/595765736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 msgr2=0x55c57e90be50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.301+0000 7f6c23400640 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90be50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.301+0000 7f6c23400640 1 -- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 msgr2=0x55c57e90b3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.301+0000 7f6c23400640 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e90b3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.301+0000 7f6c23400640 1 -- 172.21.15.57:0/595765736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e648620 con 0x55c57e903b00 2024-09-12T03:11:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.302+0000 7f6c23400640 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009900 tx=0x7f6c10003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7848195dac3e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.307+0000 7f6c0a000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:11:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.307+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10040070 con 0x55c57e903b00 2024-09-12T03:11:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.308+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f6c10003b90 con 0x55c57e903b00 2024-09-12T03:11:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.308+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c57e876830 con 0x55c57e903b00 2024-09-12T03:11:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.308+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c10035040 con 0x55c57e903b00 2024-09-12T03:11:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.309+0000 7f6c2dffa740 10 client.15519 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.309+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c57e91cc00 con 0x55c57e903b00 2024-09-12T03:11:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.311+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c57e876830 con 0x55c57e903b00 2024-09-12T03:11:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.311+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6c1003c6e0 con 0x55c57e903b00 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 10 client.15519 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 20 client.15519 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e91e0c0 0x55c57e9204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 4 client.15519 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:11:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57e870b80 con 0x55c57e91e0c0 2024-09-12T03:11:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.312+0000 7f6c23e00640 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e91e0c0 0x55c57e9204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.313+0000 7f6c23e00640 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e91e0c0 0x55c57e9204b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3c3e4dc74c4c30 server_cookie=398965f370e3d134 in_seq=0 out_seq=0 2024-09-12T03:11:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.313+0000 7f6c0a000640 10 client.15519 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.314+0000 7f6c0a000640 1 -- 172.21.15.57:0/595765736 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c57e870b80 con 0x55c57e91e0c0 2024-09-12T03:11:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.314+0000 7f6c0a000640 10 client.15519 handle_command_reply: tid=0 2024-09-12T03:11:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.314+0000 7f6c2dffa740 1 client.15519 shutdown 2024-09-12T03:11:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.315+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 msgr2=0x55c57e90b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.315+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009900 tx=0x7f6c10003820 comp rx=0 tx=0).stop 2024-09-12T03:11:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 shutdown_connections 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e91e0c0 0x55c57e9204b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e903b00 0x55c57e90b910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/595765736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e90b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 >> 172.21.15.57:0/595765736 conn(0x55c57e8ff750 msgr2=0x55c57e9054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.316+0000 7f6c0a000640 0 client.15519 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.324+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 shutdown_connections 2024-09-12T03:11:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.325+0000 7f6c2dffa740 1 -- 172.21.15.57:0/595765736 wait complete. 2024-09-12T03:11:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.330+0000 7f6c2dffa740 20 client.15519 trim_cache size 0 max 16384 2024-09-12T03:11:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.382+0000 7f6c2dffa740 1 Processor -- start 2024-09-12T03:11:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.382+0000 7f6c2dffa740 1 -- start start 2024-09-12T03:11:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e9031e0 0x55c57e8fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b8020 con 0x55c57e9031e0 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b58f0 con 0x55c57e90d770 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b3740 con 0x55c57e90e3a0 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c29200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c29200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48688/0 (socket says 172.21.15.57:48688) 2024-09-12T03:11:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.383+0000 7f6c29200640 1 -- 172.21.15.57:0/1535979131 learned_addr learned my addr 172.21.15.57:0/1535979131 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 -- 172.21.15.57:0/1535979131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 msgr2=0x55c57e90d230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 -- 172.21.15.57:0/1535979131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e9031e0 msgr2=0x55c57e8fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e9031e0 0x55c57e8fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 -- 172.21.15.57:0/1535979131 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e90a360 con 0x55c57e90d770 2024-09-12T03:11:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c29200640 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00ab10 tx=0x7f6c1c00a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66017ccfbf4df35b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c14c00640 1 -- 172.21.15.57:0/1535979131 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01e070 con 0x55c57e90d770 2024-09-12T03:11:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c14c00640 1 -- 172.21.15.57:0/1535979131 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c0248e0 con 0x55c57e90d770 2024-09-12T03:11:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.384+0000 7f6c14c00640 1 -- 172.21.15.57:0/1535979131 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c01a8b0 con 0x55c57e90d770 2024-09-12T03:11:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.385+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1535979131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 msgr2=0x55c57e925dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.385+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00ab10 tx=0x7f6c1c00a5c0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1535979131 shutdown_connections 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e90d770 0x55c57e925dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90e3a0 0x55c57e90d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1535979131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e9031e0 0x55c57e8fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1535979131 >> 172.21.15.57:0/1535979131 conn(0x55c57e8ff750 msgr2=0x55c57e90cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1535979131 shutdown_connections 2024-09-12T03:11:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.390+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1535979131 wait complete. 2024-09-12T03:11:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.391+0000 7f6c2dffa740 1 -- start start 2024-09-12T03:11:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.391+0000 7f6c2dffa740 1 Processor -- start 2024-09-12T03:11:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90d770 0x55c57e908240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e90e3a0 0x55c57e908980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b3db0 con 0x55c57e90e3a0 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b5220 con 0x55c57e9031e0 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c241b4480 con 0x55c57e90d770 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c2dffa740 10 client.0 _ll_register_callbacks cb 0x7f6c10005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.392+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48698/0 (socket says 172.21.15.57:48698) 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 -- 172.21.15.57:0/1402073844 learned_addr learned my addr 172.21.15.57:0/1402073844 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:11:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23400640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90d770 0x55c57e908240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 -- 172.21.15.57:0/1402073844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90d770 msgr2=0x55c57e908240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90d770 0x55c57e908240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 -- 172.21.15.57:0/1402073844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e90e3a0 msgr2=0x55c57e908980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e90e3a0 0x55c57e908980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.393+0000 7f6c23e00640 1 -- 172.21.15.57:0/1402073844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e998da0 con 0x55c57e9031e0 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.394+0000 7f6c23e00640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800a630 tx=0x7f6c1809b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f07de4adf0900f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.394+0000 7f6c0a000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:11:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.394+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180b9060 con 0x55c57e9031e0 2024-09-12T03:11:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.394+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f6c18099c50 con 0x55c57e9031e0 2024-09-12T03:11:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.394+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c1c0248e0 con 0x55c57e9031e0 2024-09-12T03:11:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.399+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c1801d040 con 0x55c57e9031e0 2024-09-12T03:11:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.403+0000 7f6c2dffa740 10 client.25658 fetch_fsmap learned FSMap version 30 2024-09-12T03:11:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.403+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c57e999fa0 con 0x55c57e9031e0 2024-09-12T03:11:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.405+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c1c0248e0 con 0x55c57e9031e0 2024-09-12T03:11:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.405+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6c180cf300 con 0x55c57e9031e0 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 20 client.25658 populate_metadata read hostname 'smithi057' 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e99a670 0x55c57e99ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 4 client.25658 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:11:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.406+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c57e7ed7c0 con 0x55c57e99a670 2024-09-12T03:11:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.407+0000 7f6c23400640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e99a670 0x55c57e99ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.408+0000 7f6c23400640 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e99a670 0x55c57e99ca60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7ff24ce1fa3090 server_cookie=5338ced7a0456cc1 in_seq=0 out_seq=0 2024-09-12T03:11:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.408+0000 7f6c0a000640 10 client.25658 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.431+0000 7f6c0a000640 1 -- 172.21.15.57:0/1402073844 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c57e7ed7c0 con 0x55c57e99a670 2024-09-12T03:11:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.431+0000 7f6c0a000640 10 client.25658 handle_command_reply: tid=0 2024-09-12T03:11:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.431+0000 7f6c2dffa740 1 client.25658 shutdown 2024-09-12T03:11:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 msgr2=0x55c57e916c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800a630 tx=0x7f6c1809b3c0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 shutdown_connections 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c57e99a670 0x55c57e99ca60 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c57e90e3a0 0x55c57e908980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c57e90d770 0x55c57e908240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 --2- 172.21.15.57:0/1402073844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c57e9031e0 0x55c57e916c60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.432+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 >> 172.21.15.57:0/1402073844 conn(0x55c57e8ff750 msgr2=0x55c57e904c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.433+0000 7f6c0a000640 0 client.25658 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:11:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.433+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 shutdown_connections 2024-09-12T03:11:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.433+0000 7f6c2dffa740 1 -- 172.21.15.57:0/1402073844 wait complete. 2024-09-12T03:11:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.433+0000 7f6c2dffa740 20 client.25658 trim_cache size 0 max 16384 2024-09-12T03:11:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:11:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:11:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:11:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:11:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.436+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 msgr2=0x7f6c0c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.436+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 0x7f6c0c0a53c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0155c0 tx=0x7f6c1c00f000 comp rx=0 tx=0).stop 2024-09-12T03:11:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.436+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 msgr2=0x7f6c24110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.436+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800c890 tx=0x7f6c1800cd50 comp rx=0 tx=0).stop 2024-09-12T03:11:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 shutdown_connections 2024-09-12T03:11:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c2410e0c0 0x7f6c2410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c0c0a2f00 0x7f6c0c0a53c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c241090b0 0x7f6c2410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 --2- 172.21.15.57:0/2226892671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c24103f60 0x7f6c24110ae0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 >> 172.21.15.57:0/2226892671 conn(0x7f6c240ff9b0 msgr2=0x7f6c24100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 shutdown_connections 2024-09-12T03:11:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:11:53.438+0000 7f6c2b200640 1 -- 172.21.15.57:0/2226892671 wait complete. 2024-09-12T03:11:53.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:53 smithi057 ceph-mon[30361]: pgmap v181: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 1.78k op/s 2024-09-12T03:11:53.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:53 smithi204 ceph-mon[35013]: pgmap v181: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 1.78k op/s 2024-09-12T03:11:53.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:53 smithi181 ceph-mon[35320]: pgmap v181: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 1.78k op/s 2024-09-12T03:11:55.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:55 smithi057 ceph-mon[30361]: pgmap v182: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 190 MiB/s wr, 1.23k op/s 2024-09-12T03:11:55.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:55 smithi204 ceph-mon[35013]: pgmap v182: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 190 MiB/s wr, 1.23k op/s 2024-09-12T03:11:55.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:55 smithi181 ceph-mon[35320]: pgmap v182: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 190 MiB/s wr, 1.23k op/s 2024-09-12T03:11:57.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:57 smithi057 ceph-mon[30361]: pgmap v183: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 223 MiB/s wr, 1.43k op/s 2024-09-12T03:11:57.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:57 smithi204 ceph-mon[35013]: pgmap v183: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 223 MiB/s wr, 1.43k op/s 2024-09-12T03:11:57.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:57 smithi181 ceph-mon[35320]: pgmap v183: 129 pgs: 129 active+clean; 42 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 12 KiB/s rd, 223 MiB/s wr, 1.43k op/s 2024-09-12T03:11:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:11:59 smithi057 ceph-mon[30361]: pgmap v184: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 24 KiB/s rd, 251 MiB/s wr, 1.59k op/s 2024-09-12T03:11:59.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:11:59 smithi204 ceph-mon[35013]: pgmap v184: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 24 KiB/s rd, 251 MiB/s wr, 1.59k op/s 2024-09-12T03:11:59.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:11:59 smithi181 ceph-mon[35320]: pgmap v184: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 24 KiB/s rd, 251 MiB/s wr, 1.59k op/s 2024-09-12T03:12:01.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:01 smithi057 ceph-mon[30361]: pgmap v185: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.23k op/s 2024-09-12T03:12:01.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:01 smithi204 ceph-mon[35013]: pgmap v185: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.23k op/s 2024-09-12T03:12:01.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:01 smithi181 ceph-mon[35320]: pgmap v185: 129 pgs: 129 active+clean; 43 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 17 KiB/s rd, 198 MiB/s wr, 1.23k op/s 2024-09-12T03:12:03.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:03 smithi057 ceph-mon[30361]: pgmap v186: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 23 KiB/s rd, 275 MiB/s wr, 1.68k op/s 2024-09-12T03:12:03.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:03 smithi204 ceph-mon[35013]: pgmap v186: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 23 KiB/s rd, 275 MiB/s wr, 1.68k op/s 2024-09-12T03:12:03.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:03 smithi181 ceph-mon[35320]: pgmap v186: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 23 KiB/s rd, 275 MiB/s wr, 1.68k op/s 2024-09-12T03:12:05.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:05 smithi057 ceph-mon[30361]: pgmap v187: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.16k op/s 2024-09-12T03:12:05.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:05 smithi204 ceph-mon[35013]: pgmap v187: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.16k op/s 2024-09-12T03:12:05.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:05 smithi181 ceph-mon[35320]: pgmap v187: 129 pgs: 129 active+clean; 43 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 18 KiB/s rd, 191 MiB/s wr, 1.16k op/s 2024-09-12T03:12:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.010+0000 7f1148800640 1 -- 172.21.15.57:0/1836215529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140109b10 msgr2=0x7f1140114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.010+0000 7f1148800640 1 --2- 172.21.15.57:0/1836215529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140109b10 0x7f1140114570 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1130013150 tx=0x7f1130032760 comp rx=0 tx=0).stop 2024-09-12T03:12:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 -- 172.21.15.57:0/1836215529 shutdown_connections 2024-09-12T03:12:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 --2- 172.21.15.57:0/1836215529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1140114ab0 0x7f1140116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 --2- 172.21.15.57:0/1836215529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140109b10 0x7f1140114570 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 --2- 172.21.15.57:0/1836215529 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f11401091f0 0x7f11401095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 -- 172.21.15.57:0/1836215529 >> 172.21.15.57:0/1836215529 conn(0x7f114006ce00 msgr2=0x7f114006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.011+0000 7f1148800640 1 -- 172.21.15.57:0/1836215529 shutdown_connections 2024-09-12T03:12:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.012+0000 7f1148800640 1 -- 172.21.15.57:0/1836215529 wait complete. 2024-09-12T03:12:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.013+0000 7f1148800640 1 Processor -- start 2024-09-12T03:12:07.014 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.013+0000 7f1148800640 1 -- start start 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11401091f0 0x7f11401b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1140109b10 0x7f11401b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118500 con 0x7f1140109b10 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118380 con 0x7f11401091f0 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.014+0000 7f1148800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1140118680 con 0x7f1140114ab0 2024-09-12T03:12:07.015 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.015+0000 7f1145e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11401091f0 0x7f11401b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.015+0000 7f1146800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.015+0000 7f1146800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:36910/0 (socket says 172.21.15.57:36910) 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.015+0000 7f1146800640 1 -- 172.21.15.57:0/1832909852 learned_addr learned my addr 172.21.15.57:0/1832909852 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.016+0000 7f1146800640 1 -- 172.21.15.57:0/1832909852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11401091f0 msgr2=0x7f11401b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.016+0000 7f1146800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11401091f0 0x7f11401b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.016+0000 7f1146800640 1 -- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1140109b10 msgr2=0x7f11401b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.016+0000 7f1146800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1140109b10 0x7f11401b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.016+0000 7f1146800640 1 -- 172.21.15.57:0/1832909852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401b0020 con 0x7f1140114ab0 2024-09-12T03:12:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.017+0000 7f1146800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b9e0 tx=0x7f113c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.018+0000 7f113aa00640 1 -- 172.21.15.57:0/1832909852 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c019070 con 0x7f1140114ab0 2024-09-12T03:12:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.018+0000 7f113aa00640 1 -- 172.21.15.57:0/1832909852 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c014440 con 0x7f1140114ab0 2024-09-12T03:12:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.018+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401b0310 con 0x7f1140114ab0 2024-09-12T03:12:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.019+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401bc620 con 0x7f1140114ab0 2024-09-12T03:12:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.019+0000 7f113aa00640 1 -- 172.21.15.57:0/1832909852 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0125e0 con 0x7f1140114ab0 2024-09-12T03:12:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.026+0000 7f114b5ca740 1 Processor -- start 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.027+0000 7f113aa00640 1 -- 172.21.15.57:0/1832909852 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f113c01c070 con 0x7f1140114ab0 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 -- start start 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565de1e0 0x5595565de5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565deb00 0x5595565e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x559556544100 con 0x5595565deb00 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595564ec1b0 con 0x5595565e93a0 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f114b5ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595565400c0 con 0x5595565de1e0 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.028+0000 7f1145400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.029+0000 7f1145400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38424/0 (socket says 172.21.15.57:38424) 2024-09-12T03:12:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.029+0000 7f1145400640 1 -- 172.21.15.57:0/3816841245 learned_addr learned my addr 172.21.15.57:0/3816841245 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.029+0000 7f1145400640 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565de1e0 0x5595565de5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.029+0000 7f1145e00640 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565deb00 0x5595565e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.030+0000 7f1145400640 1 -- 172.21.15.57:0/3816841245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565de1e0 msgr2=0x5595565de5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.030+0000 7f1145400640 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565de1e0 0x5595565de5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.030+0000 7f1145400640 1 -- 172.21.15.57:0/3816841245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565deb00 msgr2=0x5595565e8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.030+0000 7f1145400640 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565deb00 0x5595565e8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.030+0000 7f1145400640 1 -- 172.21.15.57:0/3816841245 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559556257210 con 0x5595565e93a0 2024-09-12T03:12:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.031+0000 7f113aa00640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 0x7f11280a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.031+0000 7f113aa00640 1 -- 172.21.15.57:0/1832909852 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f113c0cebf0 con 0x7f1140114ab0 2024-09-12T03:12:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.032+0000 7f1145400640 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1130005ab0 tx=0x7f1130005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e36eac49e02b252b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.032+0000 7f1145e00640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 0x7f11280a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.033+0000 7f112f000640 1 -- 172.21.15.57:0/3816841245 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130049070 con 0x5595565e93a0 2024-09-12T03:12:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.033+0000 7f112f000640 1 -- 172.21.15.57:0/3816841245 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11300112d0 con 0x5595565e93a0 2024-09-12T03:12:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.033+0000 7f112f000640 1 -- 172.21.15.57:0/3816841245 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11300050d0 con 0x5595565e93a0 2024-09-12T03:12:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 -- 172.21.15.57:0/3816841245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 msgr2=0x5595565eb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1130005ab0 tx=0x7f1130005a20 comp rx=0 tx=0).stop 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 -- 172.21.15.57:0/3816841245 shutdown_connections 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565eb790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565deb00 0x5595565e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 --2- 172.21.15.57:0/3816841245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565de1e0 0x5595565de5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 -- 172.21.15.57:0/3816841245 >> 172.21.15.57:0/3816841245 conn(0x5595565da750 msgr2=0x5595565dab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f1145e00640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 0x7f11280a5440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fd0 tx=0x7f1134005f20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.034+0000 7f114b5ca740 1 -- 172.21.15.57:0/3816841245 shutdown_connections 2024-09-12T03:12:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.035+0000 7f114b5ca740 1 -- 172.21.15.57:0/3816841245 wait complete. 2024-09-12T03:12:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.036+0000 7f114b5ca740 1 -- start start 2024-09-12T03:12:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.036+0000 7f114b5ca740 1 Processor -- start 2024-09-12T03:12:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.036+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.036+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f114b5ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5595565ebf70 con 0x5595565de1e0 2024-09-12T03:12:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f114b5ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5595564cd090 con 0x5595565e93a0 2024-09-12T03:12:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f114b5ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5595565ec0f0 con 0x5595565deb00 2024-09-12T03:12:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f114b5ca740 10 client.0 _ll_register_callbacks cb 0x559556521f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f1145e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f1145400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f1145400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:36918/0 (socket says 172.21.15.57:36918) 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f1145400640 1 -- 172.21.15.57:0/824329238 learned_addr learned my addr 172.21.15.57:0/824329238 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.037+0000 7f1146800640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 -- 172.21.15.57:0/824329238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 msgr2=0x5595565e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 -- 172.21.15.57:0/824329238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 msgr2=0x5595565e6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 -- 172.21.15.57:0/824329238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559556323620 con 0x5595565e93a0 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1146800640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565e6e80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f113c0cc040 tx=0x7f113c09b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedbf056b793d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1123400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.038+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0cc3c0 con 0x5595565e93a0 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11300112d0 con 0x5595565e93a0 2024-09-12T03:12:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f113c0a37f0 con 0x5595565e93a0 2024-09-12T03:12:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f113c09a260 con 0x5595565e93a0 2024-09-12T03:12:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f114b5ca740 10 client.25667 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595565f7da0 con 0x5595565e93a0 2024-09-12T03:12:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11300112d0 con 0x5595565e93a0 2024-09-12T03:12:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.039+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f113c099cd0 con 0x5595565e93a0 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 10 client.25667 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 20 client.25667 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595565f9260 0x5595565fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 4 client.25667 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559556551830 con 0x5595565f9260 2024-09-12T03:12:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.040+0000 7f1145e00640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595565f9260 0x5595565fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.043+0000 7f1145e00640 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595565f9260 0x5595565fb690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edb7c7d7c27ff25 server_cookie=227dff1971f62ee7 in_seq=0 out_seq=0 2024-09-12T03:12:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.043+0000 7f1123400640 10 client.25667 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.045+0000 7f1123400640 1 -- 172.21.15.57:0/824329238 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559556551830 con 0x5595565f9260 2024-09-12T03:12:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.045+0000 7f1123400640 10 client.25667 handle_command_reply: tid=0 2024-09-12T03:12:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.045+0000 7f114b5ca740 1 client.25667 shutdown 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 msgr2=0x5595565e6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565e6e80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f113c0cc040 tx=0x7f113c09b650 comp rx=0 tx=0).stop 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 shutdown_connections 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595565f9260 0x5595565fb690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565e6e80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 --2- 172.21.15.57:0/824329238 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f1123400640 0 client.25667 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.047+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 >> 172.21.15.57:0/824329238 conn(0x5595565da750 msgr2=0x5595565e04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.048+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 shutdown_connections 2024-09-12T03:12:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.048+0000 7f114b5ca740 1 -- 172.21.15.57:0/824329238 wait complete. 2024-09-12T03:12:07.050 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.048+0000 7f114b5ca740 20 client.25667 trim_cache size 0 max 16384 2024-09-12T03:12:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.083+0000 7f114b5ca740 1 Processor -- start 2024-09-12T03:12:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.083+0000 7f114b5ca740 1 -- start start 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f11240127e0 con 0x5595565de1e0 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f11240100b0 con 0x5595565e93a0 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.084+0000 7f114b5ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f112400df00 con 0x5595565deb00 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.085+0000 7f1146800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.085+0000 7f1146800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38434/0 (socket says 172.21.15.57:38434) 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.085+0000 7f1146800640 1 -- 172.21.15.57:0/4166428157 learned_addr learned my addr 172.21.15.57:0/4166428157 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.085+0000 7f1145400640 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 -- 172.21.15.57:0/4166428157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 msgr2=0x5595565e0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 -- 172.21.15.57:0/4166428157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 msgr2=0x5595565e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 -- 172.21.15.57:0/4166428157 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595565fbd10 con 0x5595565e93a0 2024-09-12T03:12:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f1146800640 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f113c097790 tx=0x7f113c09a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6e0b2107848d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.086+0000 7f112e600640 1 -- 172.21.15.57:0/4166428157 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0cc3c0 con 0x5595565e93a0 2024-09-12T03:12:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.087+0000 7f112e600640 1 -- 172.21.15.57:0/4166428157 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c0a05f0 con 0x5595565e93a0 2024-09-12T03:12:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.087+0000 7f112e600640 1 -- 172.21.15.57:0/4166428157 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0970b0 con 0x5595565e93a0 2024-09-12T03:12:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.088+0000 7f114b5ca740 1 -- 172.21.15.57:0/4166428157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 msgr2=0x559556600fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.088+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f113c097790 tx=0x7f113c09a7b0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.089+0000 7f114b5ca740 1 -- 172.21.15.57:0/4166428157 shutdown_connections 2024-09-12T03:12:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.089+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x559556600fa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.089+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.089+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4166428157 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.090+0000 7f114b5ca740 1 -- 172.21.15.57:0/4166428157 >> 172.21.15.57:0/4166428157 conn(0x5595565da750 msgr2=0x5595565e8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.090+0000 7f114b5ca740 1 -- 172.21.15.57:0/4166428157 shutdown_connections 2024-09-12T03:12:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.090+0000 7f114b5ca740 1 -- 172.21.15.57:0/4166428157 wait complete. 2024-09-12T03:12:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.091+0000 7f114b5ca740 1 -- start start 2024-09-12T03:12:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.091+0000 7f114b5ca740 1 Processor -- start 2024-09-12T03:12:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.092+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f114b5ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f1145400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f1145400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:36942/0 (socket says 172.21.15.57:36942) 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f1145400640 1 -- 172.21.15.57:0/4091342087 learned_addr learned my addr 172.21.15.57:0/4091342087 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f1146800640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f112400e570 con 0x5595565de1e0 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.093+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f112400f9e0 con 0x5595565e93a0 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.094+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f112400ec40 con 0x5595565deb00 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.094+0000 7f114b5ca740 10 client.0 _ll_register_callbacks cb 0x7f113c09abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.094+0000 7f1145e00640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 -- 172.21.15.57:0/4091342087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 msgr2=0x5595565f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 -- 172.21.15.57:0/4091342087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 msgr2=0x5595565f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595565e71f0 con 0x5595565e93a0 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1146800640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565ee310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f113c014e10 tx=0x7f113c099de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ef20cbfcea58e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.095+0000 7f1123400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:12:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.096+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0970b0 con 0x5595565e93a0 2024-09-12T03:12:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.096+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f113c0a05f0 con 0x5595565e93a0 2024-09-12T03:12:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.096+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9466+0+0 (secure 0 0 0) 0x7f113c00cb20 con 0x5595565e93a0 2024-09-12T03:12:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.097+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f113c0cc3c0 con 0x5595565e93a0 2024-09-12T03:12:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.098+0000 7f114b5ca740 10 client.25673 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.098+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559556664470 con 0x5595565e93a0 2024-09-12T03:12:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.098+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c0a05f0 con 0x5595565e93a0 2024-09-12T03:12:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.101+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f113c0b9c60 con 0x5595565e93a0 2024-09-12T03:12:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 10 client.25673 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 10 client.25673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 20 client.25673 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595566648f0 0x559556666ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 4 client.25673 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:12:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.102+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55955654bb80 con 0x5595566648f0 2024-09-12T03:12:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.104+0000 7f1145e00640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595566648f0 0x559556666ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.105+0000 7f1145e00640 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595566648f0 0x559556666ce0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0d2ef9b8d9e3c5 server_cookie=d9d8b30d5c471a59 in_seq=0 out_seq=0 2024-09-12T03:12:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.105+0000 7f1123400640 10 client.25673 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.120+0000 7f1123400640 1 -- 172.21.15.57:0/4091342087 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55955654bb80 con 0x5595566648f0 2024-09-12T03:12:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.120+0000 7f1123400640 10 client.25673 handle_command_reply: tid=0 2024-09-12T03:12:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.120+0000 7f114b5ca740 1 client.25673 shutdown 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 msgr2=0x5595565ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565ee310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f113c014e10 tx=0x7f113c099de0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 shutdown_connections 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5595566648f0 0x559556666ce0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5595565e93a0 0x5595565ee310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5595565deb00 0x5595565f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 --2- 172.21.15.57:0/4091342087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5595565de1e0 0x5595565f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f1123400640 0 client.25673 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.121+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 >> 172.21.15.57:0/4091342087 conn(0x5595565da750 msgr2=0x5595565e7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.122+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 shutdown_connections 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.122+0000 7f114b5ca740 1 -- 172.21.15.57:0/4091342087 wait complete. 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.122+0000 7f114b5ca740 20 client.25673 trim_cache size 0 max 16384 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:12:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.123+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 msgr2=0x7f11280a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.123+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 0x7f11280a5440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1134005fd0 tx=0x7f1134005f20 comp rx=0 tx=0).stop 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.123+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 msgr2=0x7f11401af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.123+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f113c00b9e0 tx=0x7f113c00bea0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 shutdown_connections 2024-09-12T03:12:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1140114ab0 0x7f11401af760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f11280a2f80 0x7f11280a5440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1140109b10 0x7f11401b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 --2- 172.21.15.57:0/1832909852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11401091f0 0x7f11401b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 >> 172.21.15.57:0/1832909852 conn(0x7f114006ce00 msgr2=0x7f11400707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 shutdown_connections 2024-09-12T03:12:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:07.124+0000 7f1148800640 1 -- 172.21.15.57:0/1832909852 wait complete. 2024-09-12T03:12:07.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:07 smithi057 ceph-mon[30361]: pgmap v188: 129 pgs: 129 active+clean; 44 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 1.43k op/s 2024-09-12T03:12:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:07 smithi204 ceph-mon[35013]: pgmap v188: 129 pgs: 129 active+clean; 44 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 1.43k op/s 2024-09-12T03:12:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:07 smithi181 ceph-mon[35320]: pgmap v188: 129 pgs: 129 active+clean; 44 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 28 KiB/s rd, 225 MiB/s wr, 1.43k op/s 2024-09-12T03:12:08.569 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:08 smithi057 ceph-mon[30361]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:12:08.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:08 smithi204 ceph-mon[35013]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:12:08.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:08 smithi181 ceph-mon[35320]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T03:12:09.588 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:09 smithi057 ceph-mon[30361]: pgmap v190: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 236 MiB/s wr, 1.62k op/s 2024-09-12T03:12:09.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:09 smithi204 ceph-mon[35013]: pgmap v190: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 236 MiB/s wr, 1.62k op/s 2024-09-12T03:12:09.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:09 smithi181 ceph-mon[35320]: pgmap v190: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 236 MiB/s wr, 1.62k op/s 2024-09-12T03:12:11.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:11 smithi057 ceph-mon[30361]: pgmap v191: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 238 MiB/s wr, 1.64k op/s 2024-09-12T03:12:11.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:11 smithi204 ceph-mon[35013]: pgmap v191: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 238 MiB/s wr, 1.64k op/s 2024-09-12T03:12:11.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:11 smithi181 ceph-mon[35320]: pgmap v191: 129 pgs: 129 active+clean; 44 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 29 KiB/s rd, 238 MiB/s wr, 1.64k op/s 2024-09-12T03:12:13.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:13 smithi057 ceph-mon[30361]: pgmap v192: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:13.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:13 smithi204 ceph-mon[35013]: pgmap v192: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:13.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:13 smithi181 ceph-mon[35320]: pgmap v192: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:14.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:14 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:14.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:14 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:15.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:14 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:15.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:15 smithi204 ceph-mon[35013]: pgmap v193: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:15.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:15 smithi181 ceph-mon[35320]: pgmap v193: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:16.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:15 smithi057 ceph-mon[30361]: pgmap v193: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 33 KiB/s rd, 245 MiB/s wr, 1.64k op/s 2024-09-12T03:12:17.872 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:17 smithi057 ceph-mon[30361]: pgmap v194: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 22 KiB/s rd, 249 MiB/s wr, 1.45k op/s 2024-09-12T03:12:17.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:17 smithi057 ceph-mon[30361]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:12:17.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:17 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:17 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.873 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:17 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:12:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:17 smithi204 ceph-mon[35013]: pgmap v194: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 22 KiB/s rd, 249 MiB/s wr, 1.45k op/s 2024-09-12T03:12:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:17 smithi204 ceph-mon[35013]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:12:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:17 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:17 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:17 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:17 smithi181 ceph-mon[35320]: pgmap v194: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 22 KiB/s rd, 249 MiB/s wr, 1.45k op/s 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:17 smithi181 ceph-mon[35320]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:17 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:17 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:17 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:12:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:19 smithi057 ceph-mon[30361]: pgmap v196: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 258 MiB/s wr, 1.10k op/s 2024-09-12T03:12:19.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:19 smithi204 ceph-mon[35013]: pgmap v196: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 258 MiB/s wr, 1.10k op/s 2024-09-12T03:12:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:19 smithi181 ceph-mon[35320]: pgmap v196: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 258 MiB/s wr, 1.10k op/s 2024-09-12T03:12:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 -- 172.21.15.57:0/2262970666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa681044a0 msgr2=0x7ffa681109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 --2- 172.21.15.57:0/2262970666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa681044a0 0x7ffa681109d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009cd0 tx=0x7ffa6002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 -- 172.21.15.57:0/2262970666 shutdown_connections 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 --2- 172.21.15.57:0/2262970666 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa681044a0 0x7ffa681109d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 --2- 172.21.15.57:0/2262970666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa68103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 --2- 172.21.15.57:0/2262970666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa68109ba0 0x7ffa68109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 -- 172.21.15.57:0/2262970666 >> 172.21.15.57:0/2262970666 conn(0x7ffa680ff7c0 msgr2=0x7ffa68101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 -- 172.21.15.57:0/2262970666 shutdown_connections 2024-09-12T03:12:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.277+0000 7ffa6e600640 1 -- 172.21.15.57:0/2262970666 wait complete. 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.278+0000 7ffa6e600640 1 Processor -- start 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 -- start start 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 0x7ffa6819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68113f50 con 0x7ffa68109ba0 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68113dd0 con 0x7ffa68103b00 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681140d0 con 0x7ffa681044a0 2024-09-12T03:12:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60048/0 (socket says 172.21.15.57:60048) 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6d200640 1 -- 172.21.15.57:0/4056983978 learned_addr learned my addr 172.21.15.57:0/4056983978 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43352/0 (socket says 172.21.15.57:43352) 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.279+0000 7ffa67e00640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 0x7ffa6819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 -- 172.21.15.57:0/4056983978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 msgr2=0x7ffa6819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 0x7ffa6819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 -- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 msgr2=0x7ffa6819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 -- 172.21.15.57:0/4056983978 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6819f490 con 0x7ffa68103b00 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa67e00640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 0x7ffa6819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6d200640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9f0 tx=0x7ffa5c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa65600640 1 -- 172.21.15.57:0/4056983978 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00cd80 con 0x7ffa68103b00 2024-09-12T03:12:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa65600640 1 -- 172.21.15.57:0/4056983978 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa5c005500 con 0x7ffa68103b00 2024-09-12T03:12:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6819f780 con 0x7ffa68103b00 2024-09-12T03:12:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.280+0000 7ffa65600640 1 -- 172.21.15.57:0/4056983978 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c010690 con 0x7ffa68103b00 2024-09-12T03:12:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.281+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa681a7010 con 0x7ffa68103b00 2024-09-12T03:12:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.283+0000 7ffa72c0c740 1 Processor -- start 2024-09-12T03:12:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 -- start start 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa65600640 1 -- 172.21.15.57:0/4056983978 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ffa5c0046c0 con 0x7ffa68103b00 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c81570 0x55e7a4c83960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4bdc100 con 0x55e7a4c76cd0 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4b841b0 con 0x55e7a4c81570 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4bd80c0 con 0x55e7a4c763b0 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa6d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa6d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43358/0 (socket says 172.21.15.57:43358) 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa6d200640 1 -- 172.21.15.57:0/3095848780 learned_addr learned my addr 172.21.15.57:0/3095848780 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.284+0000 7ffa67e00640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c81570 0x55e7a4c83960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 -- 172.21.15.57:0/3095848780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c81570 msgr2=0x55e7a4c83960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c81570 0x55e7a4c83960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 -- 172.21.15.57:0/3095848780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 msgr2=0x55e7a4c81030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 -- 172.21.15.57:0/3095848780 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a48ef210 con 0x55e7a4c763b0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6d200640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c76790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009ca0 tx=0x7ffa60009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2b7a70503a591c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa5a000640 1 -- 172.21.15.57:0/3095848780 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60048070 con 0x55e7a4c763b0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa5a000640 1 -- 172.21.15.57:0/3095848780 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60036070 con 0x55e7a4c763b0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa5a000640 1 -- 172.21.15.57:0/3095848780 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60004660 con 0x55e7a4c763b0 2024-09-12T03:12:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.285+0000 7ffa65600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 0x7ffa500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa65600640 1 -- 172.21.15.57:0/4056983978 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7ffa5c0cfc70 con 0x7ffa68103b00 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa6d200640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 0x7ffa500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 -- 172.21.15.57:0/3095848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 msgr2=0x55e7a4c76790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c76790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009ca0 tx=0x7ffa60009a20 comp rx=0 tx=0).stop 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 -- 172.21.15.57:0/3095848780 shutdown_connections 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c81570 0x55e7a4c83960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c81030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/3095848780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c76790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 -- 172.21.15.57:0/3095848780 >> 172.21.15.57:0/3095848780 conn(0x55e7a4c72920 msgr2=0x55e7a4c72d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 -- 172.21.15.57:0/3095848780 shutdown_connections 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa6d200640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 0x7ffa500a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c6a0 tx=0x7ffa5c0d3720 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.286+0000 7ffa72c0c740 1 -- 172.21.15.57:0/3095848780 wait complete. 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.287+0000 7ffa72c0c740 1 -- start start 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.287+0000 7ffa72c0c740 1 Processor -- start 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4c83ff0 con 0x55e7a4c763b0 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4b65090 con 0x55e7a4c76cd0 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e7a4c84170 con 0x55e7a4c81570 2024-09-12T03:12:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa72c0c740 10 client.0 _ll_register_callbacks cb 0x55e7a4bb9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43362/0 (socket says 172.21.15.57:43362) 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 -- 172.21.15.57:0/1314656370 learned_addr learned my addr 172.21.15.57:0/1314656370 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 -- 172.21.15.57:0/1314656370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 msgr2=0x55e7a4c7f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 -- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 msgr2=0x55e7a4c7ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c7ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa6dc00640 1 -- 172.21.15.57:0/1314656370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a49bb620 con 0x55e7a4c763b0 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.288+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009620 tx=0x7ffa60004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aa815e518089d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60048040 con 0x55e7a4c763b0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7ffa60003970 con 0x55e7a4c763b0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa60036070 con 0x55e7a4c763b0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa60003b70 con 0x55e7a4c763b0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa72c0c740 10 client.15537 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.289+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7a4c83ff0 con 0x55e7a4c763b0 2024-09-12T03:12:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60036070 con 0x55e7a4c763b0 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffa6004e070 con 0x55e7a4c763b0 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 10 client.15537 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 20 client.15537 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4c90fe0 0x55e7a4c933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 4 client.15537 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7a4be9830 con 0x55e7a4c90fe0 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.290+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4c90fe0 0x55e7a4c933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.291+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4c90fe0 0x55e7a4c933d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4049c7045653dce server_cookie=97d4d8f54e8ebec0 in_seq=0 out_seq=0 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.291+0000 7ffa4ea00640 10 client.15537 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1314656370 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7a4be9830 con 0x55e7a4c90fe0 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa4ea00640 10 client.15537 handle_command_reply: tid=0 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 client.15537 shutdown 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 msgr2=0x55e7a4c7e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009620 tx=0x7ffa60004260 comp rx=0 tx=0).stop 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 shutdown_connections 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4c90fe0 0x55e7a4c933d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c7ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa4ea00640 0 client.15537 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1314656370 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7e690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 >> 172.21.15.57:0/1314656370 conn(0x55e7a4c72920 msgr2=0x55e7a4c78680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 shutdown_connections 2024-09-12T03:12:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.292+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1314656370 wait complete. 2024-09-12T03:12:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.293+0000 7ffa72c0c740 20 client.15537 trim_cache size 0 max 16384 2024-09-12T03:12:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 Processor -- start 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 -- start start 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681b3af0 con 0x55e7a4c81570 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681b13c0 con 0x55e7a4c76cd0 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681af230 con 0x55e7a4c763b0 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60084/0 (socket says 172.21.15.57:60084) 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43372/0 (socket says 172.21.15.57:43372) 2024-09-12T03:12:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.308+0000 7ffa67e00640 1 -- 172.21.15.57:0/2591440739 learned_addr learned my addr 172.21.15.57:0/2591440739 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 -- 172.21.15.57:0/2591440739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 msgr2=0x55e7a4c7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 -- 172.21.15.57:0/2591440739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 msgr2=0x55e7a4c99050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 -- 172.21.15.57:0/2591440739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a4c76130 con 0x55e7a4c76cd0 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa67e00640 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa54012970 tx=0x7ffa54005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf76cd6243742b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa59600640 1 -- 172.21.15.57:0/2591440739 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540120d0 con 0x55e7a4c76cd0 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa59600640 1 -- 172.21.15.57:0/2591440739 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa54012270 con 0x55e7a4c76cd0 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.309+0000 7ffa59600640 1 -- 172.21.15.57:0/2591440739 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004a00 con 0x55e7a4c76cd0 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 -- 172.21.15.57:0/2591440739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 msgr2=0x55e7a4c98b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa54012970 tx=0x7ffa54005f90 comp rx=0 tx=0).stop 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 -- 172.21.15.57:0/2591440739 shutdown_connections 2024-09-12T03:12:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c81570 0x55e7a4c99050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c98b10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/2591440739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c763b0 0x55e7a4c7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 -- 172.21.15.57:0/2591440739 >> 172.21.15.57:0/2591440739 conn(0x55e7a4c72920 msgr2=0x55e7a4c7cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 -- 172.21.15.57:0/2591440739 shutdown_connections 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.310+0000 7ffa72c0c740 1 -- 172.21.15.57:0/2591440739 wait complete. 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 -- start start 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 Processor -- start 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c89450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681af880 con 0x55e7a4c763b0 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681b0cf0 con 0x55e7a4c76cd0 2024-09-12T03:12:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681aff50 con 0x55e7a4c81570 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa72c0c740 10 client.0 _ll_register_callbacks cb 0x7ffa60046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa6dc00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c89450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60086/0 (socket says 172.21.15.57:60086) 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.311+0000 7ffa67e00640 1 -- 172.21.15.57:0/1232528151 learned_addr learned my addr 172.21.15.57:0/1232528151 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 -- 172.21.15.57:0/1232528151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 msgr2=0x55e7a4c7aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 -- 172.21.15.57:0/1232528151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 msgr2=0x55e7a4c89450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c89450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 -- 172.21.15.57:0/1232528151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a4c7b000 con 0x55e7a4c76cd0 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c89450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa67e00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa54012420 tx=0x7ffa54012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94b4f69996c463 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54037060 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa54012270 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.312+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7ffa5400fd20 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa54004050 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 10 client.25691 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e7a4cfb290 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa54012270 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffa54003bf0 con 0x55e7a4c76cd0 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 10 client.25691 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 20 client.25691 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4cfb9f0 0x55e7a4cfde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 4 client.25691 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:12:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7a4be3b80 con 0x55e7a4cfb9f0 2024-09-12T03:12:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.313+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4cfb9f0 0x55e7a4cfde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.314+0000 7ffa6dc00640 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4cfb9f0 0x55e7a4cfde10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8c79c8d8f0c28 server_cookie=e1ebb91b6de00cad in_seq=0 out_seq=0 2024-09-12T03:12:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.314+0000 7ffa4ea00640 10 client.25691 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa4ea00640 1 -- 172.21.15.57:0/1232528151 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7a4be3b80 con 0x55e7a4cfb9f0 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa4ea00640 10 client.25691 handle_command_reply: tid=0 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 client.25691 shutdown 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 msgr2=0x55e7a4c89990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa54012420 tx=0x7ffa54012a80 comp rx=0 tx=0).stop 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 shutdown_connections 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e7a4cfb9f0 0x55e7a4cfde10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e7a4c81570 0x55e7a4c7aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e7a4c76cd0 0x55e7a4c89990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 --2- 172.21.15.57:0/1232528151 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e7a4c763b0 0x55e7a4c89450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 >> 172.21.15.57:0/1232528151 conn(0x55e7a4c72920 msgr2=0x55e7a4c7d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa4ea00640 0 client.25691 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 shutdown_connections 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 1 -- 172.21.15.57:0/1232528151 wait complete. 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.323+0000 7ffa72c0c740 20 client.25691 trim_cache size 0 max 16384 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:12:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 msgr2=0x7ffa500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 0x7ffa500a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c6a0 tx=0x7ffa5c0d3720 comp rx=0 tx=0).stop 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 msgr2=0x7ffa6819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00e9f0 tx=0x7ffa5c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 shutdown_connections 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ffa500a2cf0 0x7ffa500a51b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ffa68109ba0 0x7ffa6819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa681044a0 0x7ffa6819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 --2- 172.21.15.57:0/4056983978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa68103b00 0x7ffa6819a4a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 >> 172.21.15.57:0/4056983978 conn(0x7ffa680ff7c0 msgr2=0x7ffa68100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 shutdown_connections 2024-09-12T03:12:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:20.324+0000 7ffa6e600640 1 -- 172.21.15.57:0/4056983978 wait complete. 2024-09-12T03:12:21.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:21 smithi057 ceph-mon[30361]: pgmap v197: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 1.09k op/s 2024-09-12T03:12:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:21 smithi204 ceph-mon[35013]: pgmap v197: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 1.09k op/s 2024-09-12T03:12:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:21 smithi181 ceph-mon[35320]: pgmap v197: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 260 MiB/s wr, 1.09k op/s 2024-09-12T03:12:23.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:23 smithi057 ceph-mon[30361]: pgmap v198: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:23 smithi204 ceph-mon[35013]: pgmap v198: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:23 smithi181 ceph-mon[35320]: pgmap v198: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:25.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:25 smithi057 ceph-mon[30361]: pgmap v199: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:25.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:25 smithi204 ceph-mon[35013]: pgmap v199: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:25 smithi181 ceph-mon[35320]: pgmap v199: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 205 MiB/s wr, 798 op/s 2024-09-12T03:12:27.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:27 smithi204 ceph-mon[35013]: pgmap v200: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 160 MiB/s wr, 668 op/s 2024-09-12T03:12:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:27 smithi181 ceph-mon[35320]: pgmap v200: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 160 MiB/s wr, 668 op/s 2024-09-12T03:12:28.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:27 smithi057 ceph-mon[30361]: pgmap v200: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 160 MiB/s wr, 668 op/s 2024-09-12T03:12:29.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:29 smithi204 ceph-mon[35013]: pgmap v201: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 14 KiB/s rd, 140 MiB/s wr, 587 op/s 2024-09-12T03:12:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:29 smithi181 ceph-mon[35320]: pgmap v201: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 14 KiB/s rd, 140 MiB/s wr, 587 op/s 2024-09-12T03:12:30.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:29 smithi057 ceph-mon[30361]: pgmap v201: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 14 KiB/s rd, 140 MiB/s wr, 587 op/s 2024-09-12T03:12:31.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:31 smithi204 ceph-mon[35013]: pgmap v202: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 62 MiB/s wr, 359 op/s 2024-09-12T03:12:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:31 smithi181 ceph-mon[35320]: pgmap v202: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 62 MiB/s wr, 359 op/s 2024-09-12T03:12:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:31 smithi057 ceph-mon[30361]: pgmap v202: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 62 MiB/s wr, 359 op/s 2024-09-12T03:12:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.452+0000 7f7cc8a00640 1 -- 172.21.15.57:0/3245904388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4107ad0 msgr2=0x7f7cc4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.452+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/3245904388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4107ad0 0x7f7cc4107eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009900 tx=0x7f7cb402a410 comp rx=0 tx=0).stop 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 -- 172.21.15.57:0/3245904388 shutdown_connections 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/3245904388 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/3245904388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4101ac0 0x7f7cc4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/3245904388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4107ad0 0x7f7cc4107eb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 -- 172.21.15.57:0/3245904388 >> 172.21.15.57:0/3245904388 conn(0x7f7cc40fd7e0 msgr2=0x7f7cc40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 -- 172.21.15.57:0/3245904388 shutdown_connections 2024-09-12T03:12:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.453+0000 7f7cc8a00640 1 -- 172.21.15.57:0/3245904388 wait complete. 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 Processor -- start 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 -- start start 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4107ad0 0x7f7cc419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4111f60 con 0x7f7cc4102460 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4111de0 con 0x7f7cc4101ac0 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41120e0 con 0x7f7cc4107ad0 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.454+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cc2800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4107ad0 0x7f7cc419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:40550/0 (socket says 172.21.15.57:40550) 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cc1e00640 1 -- 172.21.15.57:0/2363297435 learned_addr learned my addr 172.21.15.57:0/2363297435 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 -- 172.21.15.57:0/2363297435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4107ad0 msgr2=0x7f7cc419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4107ad0 0x7f7cc419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 -- 172.21.15.57:0/2363297435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 msgr2=0x7f7cc419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 -- 172.21.15.57:0/2363297435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc419f530 con 0x7f7cc4102460 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.455+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc419aa80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00e9f0 tx=0x7f7cac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.456+0000 7f7cbaa00640 1 -- 172.21.15.57:0/2363297435 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00ccc0 con 0x7f7cc4102460 2024-09-12T03:12:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.456+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc419f820 con 0x7f7cc4102460 2024-09-12T03:12:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.456+0000 7f7cbaa00640 1 -- 172.21.15.57:0/2363297435 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac005020 con 0x7f7cc4102460 2024-09-12T03:12:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.456+0000 7f7cbaa00640 1 -- 172.21.15.57:0/2363297435 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0049e0 con 0x7f7cc4102460 2024-09-12T03:12:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.456+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc41a70b0 con 0x7f7cc4102460 2024-09-12T03:12:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 Processor -- start 2024-09-12T03:12:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7cbaa00640 1 -- 172.21.15.57:0/2363297435 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f7cac005560 con 0x7f7cc4102460 2024-09-12T03:12:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 -- start start 2024-09-12T03:12:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f7b00 0x55ac77401e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77404790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ac7735d100 con 0x55ac773f71e0 2024-09-12T03:12:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac773051b0 con 0x55ac774023a0 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.459+0000 7f7ccb829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac773590c0 con 0x55ac773f7b00 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37722/0 (socket says 172.21.15.57:37722) 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 -- 172.21.15.57:0/619155512 learned_addr learned my addr 172.21.15.57:0/619155512 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f7b00 0x55ac77401e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77404790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 -- 172.21.15.57:0/619155512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f7b00 msgr2=0x55ac77401e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f7b00 0x55ac77401e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 -- 172.21.15.57:0/619155512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 msgr2=0x55ac77404790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77404790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 -- 172.21.15.57:0/619155512 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac77070210 con 0x55ac773f71e0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cbaa00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 0x7f7ca80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.460+0000 7f7cc2800640 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009f50 tx=0x7f7cbc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f1643d6a12d8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cbaa00640 1 -- 172.21.15.57:0/2363297435 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f7cac0cf7b0 con 0x7f7cc4102460 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 0x7f7ca80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cb3000640 1 -- 172.21.15.57:0/619155512 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc004d20 con 0x55ac773f71e0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cb3000640 1 -- 172.21.15.57:0/619155512 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cbc004050 con 0x55ac773f71e0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cb3000640 1 -- 172.21.15.57:0/619155512 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc005220 con 0x55ac773f71e0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7ccb829740 1 -- 172.21.15.57:0/619155512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 msgr2=0x55ac773f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7ccb829740 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009f50 tx=0x7f7cbc0118d0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.461+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 0x7f7ca80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40096c0 tx=0x7f7cb40046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 -- 172.21.15.57:0/619155512 shutdown_connections 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77404790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f7b00 0x55ac77401e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 --2- 172.21.15.57:0/619155512 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac773f75c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 -- 172.21.15.57:0/619155512 >> 172.21.15.57:0/619155512 conn(0x55ac773f3750 msgr2=0x55ac773f3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 -- 172.21.15.57:0/619155512 shutdown_connections 2024-09-12T03:12:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 -- 172.21.15.57:0/619155512 wait complete. 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 -- start start 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.462+0000 7f7ccb829740 1 Processor -- start 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac773ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac773ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55ac77404fc0 con 0x55ac773f7b00 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55ac772e6090 con 0x55ac774023a0 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ac77405140 con 0x55ac773f71e0 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7ccb829740 10 client.0 _ll_register_callbacks cb 0x55ac7733af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37738/0 (socket says 172.21.15.57:37738) 2024-09-12T03:12:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7cbbe00640 1 -- 172.21.15.57:0/1813235348 learned_addr learned my addr 172.21.15.57:0/1813235348 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac773ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.463+0000 7f7cc2800640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac773ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 -- 172.21.15.57:0/1813235348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 msgr2=0x55ac773ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac773ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 -- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 msgr2=0x55ac773ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac773ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 -- 172.21.15.57:0/1813235348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7713c620 con 0x55ac773f7b00 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cc2800640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac773ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09b560 tx=0x7f7cac09b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b9db62c4fdcaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ca7400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac01c070 con 0x55ac773f7b00 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cbc004050 con 0x55ac773f7b00 2024-09-12T03:12:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f7cac0a4040 con 0x55ac773f7b00 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7cac0ccf10 con 0x55ac773f7b00 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ccb829740 10 client.15555 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.464+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac77410c30 con 0x55ac773f7b00 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cbc004050 con 0x55ac773f7b00 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7cac0b3070 con 0x55ac773f7b00 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 10 client.15555 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 20 client.15555 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac774120a0 0x55ac77414490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 4 client.15555 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac7736a830 con 0x55ac774120a0 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.465+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac774120a0 0x55ac77414490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.466+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac774120a0 0x55ac77414490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e688f4c4bc6e07 server_cookie=49fb87c11930e3d in_seq=0 out_seq=0 2024-09-12T03:12:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.466+0000 7f7ca7400640 10 client.15555 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ca7400640 1 -- 172.21.15.57:0/1813235348 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac7736a830 con 0x55ac774120a0 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ca7400640 10 client.15555 handle_command_reply: tid=0 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 client.15555 shutdown 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 msgr2=0x55ac773ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7cac09b560 tx=0x7f7cac09b630 comp rx=0 tx=0).stop 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 shutdown_connections 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac774120a0 0x55ac77414490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac773ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac773ff800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 --2- 172.21.15.57:0/1813235348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac773ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ca7400640 0 client.15555 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 >> 172.21.15.57:0/1813235348 conn(0x55ac773f3750 msgr2=0x55ac773f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 shutdown_connections 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 1 -- 172.21.15.57:0/1813235348 wait complete. 2024-09-12T03:12:33.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.467+0000 7f7ccb829740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T03:12:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 Processor -- start 2024-09-12T03:12:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 -- start start 2024-09-12T03:12:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac773f7b00 0x55ac77419ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac774023a0 0x55ac7741a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b3ab0 con 0x55ac773f71e0 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b1380 con 0x55ac773f7b00 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.483+0000 7f7ccb829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41af1f0 con 0x55ac774023a0 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37744/0 (socket says 172.21.15.57:37744) 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 -- 172.21.15.57:0/2160702663 learned_addr learned my addr 172.21.15.57:0/2160702663 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac773f7b00 0x55ac77419ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc2800640 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac774023a0 0x55ac7741a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 -- 172.21.15.57:0/2160702663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac774023a0 msgr2=0x55ac7741a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac774023a0 0x55ac7741a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 -- 172.21.15.57:0/2160702663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac773f7b00 msgr2=0x55ac77419ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac773f7b00 0x55ac77419ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 -- 172.21.15.57:0/2160702663 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac773f7010 con 0x55ac773f71e0 2024-09-12T03:12:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.484+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402aa00 tx=0x7f7cb403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c126e2e6bb8002a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7cb2600640 1 -- 172.21.15.57:0/2160702663 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4035040 con 0x55ac773f71e0 2024-09-12T03:12:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7cb2600640 1 -- 172.21.15.57:0/2160702663 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb4003c40 con 0x55ac773f71e0 2024-09-12T03:12:33.487 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7cb2600640 1 -- 172.21.15.57:0/2160702663 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb403d870 con 0x55ac773f71e0 2024-09-12T03:12:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 -- 172.21.15.57:0/2160702663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 msgr2=0x55ac774010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402aa00 tx=0x7f7cb403f040 comp rx=0 tx=0).stop 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 -- 172.21.15.57:0/2160702663 shutdown_connections 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac774023a0 0x55ac7741a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac773f7b00 0x55ac77419ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 --2- 172.21.15.57:0/2160702663 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f71e0 0x55ac774010b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 -- 172.21.15.57:0/2160702663 >> 172.21.15.57:0/2160702663 conn(0x55ac773f3750 msgr2=0x55ac774009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 -- 172.21.15.57:0/2160702663 shutdown_connections 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.485+0000 7f7ccb829740 1 -- 172.21.15.57:0/2160702663 wait complete. 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 -- start start 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 Processor -- start 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac77405b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77474c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41af840 con 0x55ac773f7b00 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b0cb0 con 0x55ac774023a0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41aff10 con 0x55ac773f71e0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.486+0000 7f7ccb829740 10 client.0 _ll_register_callbacks cb 0x7f7cac0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cbbe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:37752/0 (socket says 172.21.15.57:37752) 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cbbe00640 1 -- 172.21.15.57:0/3242059214 learned_addr learned my addr 172.21.15.57:0/3242059214 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac77405b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc2800640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77474c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 -- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 msgr2=0x55ac77474c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77474c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 -- 172.21.15.57:0/3242059214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 msgr2=0x55ac77406040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 -- 172.21.15.57:0/3242059214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac773eed60 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7cc1e00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac77405b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402aa00 tx=0x7f7cb402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfa1d8da76c9573 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.487+0000 7f7ca7400640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4060070 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb4003c40 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f7cb4031540 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7cb40035f0 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ccb829740 10 client.25729 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac7747c470 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.488+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb4003c40 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7cb4044070 con 0x55ac773f71e0 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 10 client.25729 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 20 client.25729 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac7747cb10 0x55ac7747ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 4 client.25729 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac77364b80 con 0x55ac7747cb10 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac7747cb10 0x55ac7747ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7cbbe00640 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac7747cb10 0x55ac7747ef30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604d0621c0689535 server_cookie=fd4f47d8cab36ab in_seq=0 out_seq=0 2024-09-12T03:12:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.489+0000 7f7ca7400640 10 client.25729 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ca7400640 1 -- 172.21.15.57:0/3242059214 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac77364b80 con 0x55ac7747cb10 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ca7400640 10 client.25729 handle_command_reply: tid=0 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 client.25729 shutdown 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 msgr2=0x55ac77405b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac77405b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7cb402aa00 tx=0x7f7cb402a410 comp rx=0 tx=0).stop 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 shutdown_connections 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55ac7747cb10 0x55ac7747ef30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ac774023a0 0x55ac77474c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55ac773f7b00 0x55ac77406040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 --2- 172.21.15.57:0/3242059214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ac773f71e0 0x55ac77405b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ca7400640 0 client.25729 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 >> 172.21.15.57:0/3242059214 conn(0x55ac773f3750 msgr2=0x55ac773f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 shutdown_connections 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 1 -- 172.21.15.57:0/3242059214 wait complete. 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.498+0000 7f7ccb829740 20 client.25729 trim_cache size 0 max 16384 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:12:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 msgr2=0x7f7ca80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 0x7f7ca80a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40096c0 tx=0x7f7cb40046c0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 msgr2=0x7f7cc419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc419aa80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00e9f0 tx=0x7f7cac00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 shutdown_connections 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc4107ad0 0x7f7cc419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7ca80a2e20 0x7f7ca80a52e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cc4102460 0x7f7cc419aa80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 --2- 172.21.15.57:0/2363297435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc4101ac0 0x7f7cc419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 >> 172.21.15.57:0/2363297435 conn(0x7f7cc40fd7e0 msgr2=0x7f7cc410d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 shutdown_connections 2024-09-12T03:12:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:33.499+0000 7f7cc8a00640 1 -- 172.21.15.57:0/2363297435 wait complete. 2024-09-12T03:12:33.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:33 smithi204 ceph-mon[35013]: pgmap v203: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 48 MiB/s wr, 299 op/s 2024-09-12T03:12:33.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:33 smithi181 ceph-mon[35320]: pgmap v203: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 48 MiB/s wr, 299 op/s 2024-09-12T03:12:34.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:33 smithi057 ceph-mon[30361]: pgmap v203: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 48 MiB/s wr, 299 op/s 2024-09-12T03:12:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:35 smithi181 ceph-mon[35320]: pgmap v204: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-12T03:12:36.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:35 smithi057 ceph-mon[30361]: pgmap v204: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-12T03:12:36.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:35 smithi204 ceph-mon[35013]: pgmap v204: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-12T03:12:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:37 smithi181 ceph-mon[35320]: pgmap v205: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:12:38.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:37 smithi057 ceph-mon[30361]: pgmap v205: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:12:38.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:37 smithi204 ceph-mon[35013]: pgmap v205: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.2 KiB/s wr, 7 op/s 2024-09-12T03:12:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:39 smithi181 ceph-mon[35320]: pgmap v206: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 9 op/s 2024-09-12T03:12:40.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:39 smithi057 ceph-mon[30361]: pgmap v206: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 9 op/s 2024-09-12T03:12:40.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:39 smithi204 ceph-mon[35013]: pgmap v206: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.5 KiB/s wr, 9 op/s 2024-09-12T03:12:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:41 smithi181 ceph-mon[35320]: pgmap v207: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:12:42.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:41 smithi057 ceph-mon[30361]: pgmap v207: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:12:42.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:41 smithi204 ceph-mon[35013]: pgmap v207: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-09-12T03:12:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:43 smithi181 ceph-mon[35320]: pgmap v208: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:12:44.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:43 smithi057 ceph-mon[30361]: pgmap v208: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:12:44.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:43 smithi204 ceph-mon[35013]: pgmap v208: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-09-12T03:12:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:45 smithi181 ceph-mon[35320]: pgmap v209: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:12:46.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:45 smithi057 ceph-mon[30361]: pgmap v209: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:12:46.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:45 smithi204 ceph-mon[35013]: pgmap v209: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.4 KiB/s wr, 6 op/s 2024-09-12T03:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 -- 172.21.15.57:0/3589680221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770103f30 msgr2=0x7f9770104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 --2- 172.21.15.57:0/3589680221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770103f30 0x7f9770104310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9760009860 tx=0x7f976002a340 comp rx=0 tx=0).stop 2024-09-12T03:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 -- 172.21.15.57:0/3589680221 shutdown_connections 2024-09-12T03:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 --2- 172.21.15.57:0/3589680221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770109080 0x7f9770110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 --2- 172.21.15.57:0/3589680221 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97701048e0 0x7f9770108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 --2- 172.21.15.57:0/3589680221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770103f30 0x7f9770104310 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 -- 172.21.15.57:0/3589680221 >> 172.21.15.57:0/3589680221 conn(0x7f97700ff980 msgr2=0x7f9770101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 -- 172.21.15.57:0/3589680221 shutdown_connections 2024-09-12T03:12:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.626+0000 7f9775400640 1 -- 172.21.15.57:0/3589680221 wait complete. 2024-09-12T03:12:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.627+0000 7f9775400640 1 Processor -- start 2024-09-12T03:12:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.627+0000 7f9775400640 1 -- start start 2024-09-12T03:12:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 0x7f977019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f977019ee30 0x7f977019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97701140c0 con 0x7f977019ee30 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9770113f40 con 0x7f9770109080 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9775400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9770114240 con 0x7f9770103f30 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f977019ee30 0x7f977019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9767e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9767e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35836/0 (socket says 172.21.15.57:35836) 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f9767e00640 1 -- 172.21.15.57:0/1008760265 learned_addr learned my addr 172.21.15.57:0/1008760265 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.628+0000 7f976e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 0x7f977019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 -- 172.21.15.57:0/1008760265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 msgr2=0x7f977019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 -- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f977019ee30 msgr2=0x7f977019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f977019ee30 0x7f977019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 -- 172.21.15.57:0/1008760265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977019f920 con 0x7f9770103f30 2024-09-12T03:12:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f9767e00640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f976e800640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 0x7f977019a570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f976002a860 tx=0x7f9760033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f9767400640 1 -- 172.21.15.57:0/1008760265 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760037040 con 0x7f9770103f30 2024-09-12T03:12:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f9767400640 1 -- 172.21.15.57:0/1008760265 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760033d50 con 0x7f9770103f30 2024-09-12T03:12:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977019fbb0 con 0x7f9770103f30 2024-09-12T03:12:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.629+0000 7f9767400640 1 -- 172.21.15.57:0/1008760265 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760003810 con 0x7f9770103f30 2024-09-12T03:12:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.630+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97701a73d0 con 0x7f9770103f30 2024-09-12T03:12:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.632+0000 7f9778480740 1 Processor -- start 2024-09-12T03:12:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 -- start start 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9767400640 1 -- 172.21.15.57:0/1008760265 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f9760042020 con 0x7f9770103f30 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556e8d34c100 con 0x556e8d3e61e0 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e8d2f41b0 con 0x556e8d3f13a0 2024-09-12T03:12:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9778480740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556e8d3480c0 con 0x556e8d3e6b00 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f976e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42826/0 (socket says 172.21.15.57:42826) 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f976f200640 1 -- 172.21.15.57:0/3282789115 learned_addr learned my addr 172.21.15.57:0/3282789115 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.633+0000 7f9767e00640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 -- 172.21.15.57:0/3282789115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 msgr2=0x556e8d3f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 -- 172.21.15.57:0/3282789115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 msgr2=0x556e8d3f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 -- 172.21.15.57:0/3282789115 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8d05f210 con 0x556e8d3e61e0 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976e800640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f9767e00640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f976f200640 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9768009f50 tx=0x7f97680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a0c460eef711f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f975f400640 1 -- 172.21.15.57:0/3282789115 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768004d20 con 0x556e8d3e61e0 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f975f400640 1 -- 172.21.15.57:0/3282789115 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9768004050 con 0x556e8d3e61e0 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.634+0000 7f975f400640 1 -- 172.21.15.57:0/3282789115 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768005280 con 0x556e8d3e61e0 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9767400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 0x7f97540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9767400640 1 -- 172.21.15.57:0/1008760265 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f97600eec40 con 0x7f9770103f30 2024-09-12T03:12:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f976e800640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 0x7f97540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 -- 172.21.15.57:0/3282789115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 msgr2=0x556e8d3e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9768009f50 tx=0x7f97680118d0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 -- 172.21.15.57:0/3282789115 shutdown_connections 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 --2- 172.21.15.57:0/3282789115 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d3e65c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.635+0000 7f9778480740 1 -- 172.21.15.57:0/3282789115 >> 172.21.15.57:0/3282789115 conn(0x556e8d3e2750 msgr2=0x556e8d3e2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.636+0000 7f9778480740 1 -- 172.21.15.57:0/3282789115 shutdown_connections 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.636+0000 7f9778480740 1 -- 172.21.15.57:0/3282789115 wait complete. 2024-09-12T03:12:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.636+0000 7f976e800640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 0x7f97540a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x556e8d3e35b0 tx=0x7f97600f26f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.636+0000 7f9778480740 1 -- start start 2024-09-12T03:12:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.636+0000 7f9778480740 1 Processor -- start 2024-09-12T03:12:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e61e0 0x556e8d3ee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 0x556e8d3ee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556e8d3f3f70 con 0x556e8d3f13a0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556e8d2d5090 con 0x556e8d3e6b00 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556e8d3f40f0 con 0x556e8d3e61e0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9778480740 10 client.0 _ll_register_callbacks cb 0x556e8d329f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f9767e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 0x556e8d3ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42830/0 (socket says 172.21.15.57:42830) 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f976f200640 1 -- 172.21.15.57:0/3588675006 learned_addr learned my addr 172.21.15.57:0/3588675006 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.637+0000 7f976f200640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e61e0 0x556e8d3ee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 -- 172.21.15.57:0/3588675006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e61e0 msgr2=0x556e8d3ee2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e61e0 0x556e8d3ee2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 -- 172.21.15.57:0/3588675006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 msgr2=0x556e8d3eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 -- 172.21.15.57:0/3588675006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8d12b620 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f976f200640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9767e00640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 0x556e8d3ee820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9758011ad0 tx=0x7f9758011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af97cb99ef1369d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9753e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800fdd0 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768004050 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.638+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f9758004030 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9758025050 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e8d3ffbc0 con 0x556e8d3e6b00 2024-09-12T03:12:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9768004050 con 0x556e8d3e6b00 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9758019070 con 0x556e8d3e6b00 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 20 client.25721 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d4010f0 0x556e8d4034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 4 client.25721 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.639+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8d359830 con 0x556e8d4010f0 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.640+0000 7f976f200640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d4010f0 0x556e8d4034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.640+0000 7f976f200640 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d4010f0 0x556e8d4034e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954b27079cb48359 server_cookie=d1d055fc423c7832 in_seq=0 out_seq=0 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.640+0000 7f9753e00640 10 client.25721 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9753e00640 1 -- 172.21.15.57:0/3588675006 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e8d359830 con 0x556e8d4010f0 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9753e00640 10 client.25721 handle_command_reply: tid=0 2024-09-12T03:12:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 client.25721 shutdown 2024-09-12T03:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 msgr2=0x556e8d3ee820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 0x556e8d3ee820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9758011ad0 tx=0x7f9758011f90 comp rx=0 tx=0).stop 2024-09-12T03:12:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 shutdown_connections 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d4010f0 0x556e8d4034e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e6b00 0x556e8d3ee820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9753e00640 0 client.25721 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 --2- 172.21.15.57:0/3588675006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e61e0 0x556e8d3ee2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 >> 172.21.15.57:0/3588675006 conn(0x556e8d3e2750 msgr2=0x556e8d3e8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 shutdown_connections 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.641+0000 7f9778480740 1 -- 172.21.15.57:0/3588675006 wait complete. 2024-09-12T03:12:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.642+0000 7f9778480740 20 client.25721 trim_cache size 0 max 16384 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 Processor -- start 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 -- start start 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d4092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b3b30 con 0x556e8d3e61e0 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b1400 con 0x556e8d3f13a0 2024-09-12T03:12:46.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f9778480740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97701af270 con 0x556e8d3e6b00 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.657+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42832/0 (socket says 172.21.15.57:42832) 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 -- 172.21.15.57:0/1825864162 learned_addr learned my addr 172.21.15.57:0/1825864162 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f9767e00640 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d4092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 -- 172.21.15.57:0/1825864162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 msgr2=0x556e8d4092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d4092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 -- 172.21.15.57:0/1825864162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 msgr2=0x556e8d3e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 -- 172.21.15.57:0/1825864162 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8d3e6010 con 0x556e8d3e61e0 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f976f200640 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9768009270 tx=0x7f9768009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fce15e8e6c95519 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f975ea00640 1 -- 172.21.15.57:0/1825864162 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768004370 con 0x556e8d3e61e0 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f975ea00640 1 -- 172.21.15.57:0/1825864162 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9768004510 con 0x556e8d3e61e0 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.658+0000 7f975ea00640 1 -- 172.21.15.57:0/1825864162 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976800f940 con 0x556e8d3e61e0 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 -- 172.21.15.57:0/1825864162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 msgr2=0x556e8d408d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9768009270 tx=0x7f9768009500 comp rx=0 tx=0).stop 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 -- 172.21.15.57:0/1825864162 shutdown_connections 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3f13a0 0x556e8d3e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d4092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 --2- 172.21.15.57:0/1825864162 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3e61e0 0x556e8d408d60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 -- 172.21.15.57:0/1825864162 >> 172.21.15.57:0/1825864162 conn(0x556e8d3e2750 msgr2=0x556e8d3e95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 -- 172.21.15.57:0/1825864162 shutdown_connections 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.659+0000 7f9778480740 1 -- 172.21.15.57:0/1825864162 wait complete. 2024-09-12T03:12:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 -- start start 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 Processor -- start 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e61e0 0x556e8d3f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97701af8c0 con 0x556e8d3f13a0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97701b0d30 con 0x556e8d3e61e0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97701aff90 con 0x556e8d3e6b00 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9778480740 10 client.0 _ll_register_callbacks cb 0x7f9758009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f976f200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e61e0 0x556e8d3f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9767e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f976f200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9767e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:33250/0 (socket says 172.21.15.57:33250) 2024-09-12T03:12:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.660+0000 7f9767e00640 1 -- 172.21.15.57:0/1935234318 learned_addr learned my addr 172.21.15.57:0/1935234318 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 -- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e61e0 msgr2=0x556e8d3f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e61e0 0x556e8d3f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 -- 172.21.15.57:0/1935234318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 msgr2=0x556e8d3f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 -- 172.21.15.57:0/1935234318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8d3e33e0 con 0x556e8d3e6b00 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9767e00640 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f975801a5b0 tx=0x7f9758011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69a3d8bfe1489c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9753e00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975802d020 con 0x556e8d3e6b00 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f97580052e0 con 0x556e8d3e6b00 2024-09-12T03:12:46.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.661+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768004510 con 0x556e8d3e6b00 2024-09-12T03:12:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9758038050 con 0x556e8d3e6b00 2024-09-12T03:12:46.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 10 client.25747 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e8d3ddd60 con 0x556e8d3e6b00 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9768004510 con 0x556e8d3e6b00 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9758019050 con 0x556e8d3e6b00 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 10 client.25747 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 20 client.25747 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d46be00 0x556e8d46e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 4 client.25747 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.662+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e8d353b80 con 0x556e8d46be00 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.663+0000 7f976f200640 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d46be00 0x556e8d46e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.663+0000 7f976f200640 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d46be00 0x556e8d46e220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71457f154e207926 server_cookie=7cfdbcf2f90abfb2 in_seq=0 out_seq=0 2024-09-12T03:12:46.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.663+0000 7f9753e00640 10 client.25747 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.667+0000 7f9753e00640 1 -- 172.21.15.57:0/1935234318 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e8d353b80 con 0x556e8d46be00 2024-09-12T03:12:46.668 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.667+0000 7f9753e00640 10 client.25747 handle_command_reply: tid=0 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.667+0000 7f9778480740 1 client.25747 shutdown 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.667+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 msgr2=0x556e8d3f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.667+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f975801a5b0 tx=0x7f9758011a10 comp rx=0 tx=0).stop 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 shutdown_connections 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556e8d46be00 0x556e8d46e220 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556e8d3f13a0 0x556e8d3f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556e8d3e6b00 0x556e8d3f9c40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 --2- 172.21.15.57:0/1935234318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556e8d3e61e0 0x556e8d3f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9753e00640 0 client.25747 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 >> 172.21.15.57:0/1935234318 conn(0x556e8d3e2750 msgr2=0x556e8d3e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 shutdown_connections 2024-09-12T03:12:46.669 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 1 -- 172.21.15.57:0/1935234318 wait complete. 2024-09-12T03:12:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9778480740 20 client.25747 trim_cache size 0 max 16384 2024-09-12T03:12:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:12:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:12:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:12:46.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 msgr2=0x7f97540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 0x7f97540a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x556e8d3e35b0 tx=0x7f97600f26f0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 msgr2=0x7f977019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.668+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 0x7f977019a570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f976002a860 tx=0x7f9760033b60 comp rx=0 tx=0).stop 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 shutdown_connections 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97540a2e20 0x7f97540a52e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f977019ee30 0x7f977019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9770109080 0x7f977019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 --2- 172.21.15.57:0/1008760265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9770103f30 0x7f977019a570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 >> 172.21.15.57:0/1008760265 conn(0x7f97700ff980 msgr2=0x7f9770100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 shutdown_connections 2024-09-12T03:12:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:46.669+0000 7f9775400640 1 -- 172.21.15.57:0/1008760265 wait complete. 2024-09-12T03:12:47.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:47 smithi181 ceph-mon[35320]: pgmap v210: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:12:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:47 smithi057 ceph-mon[30361]: pgmap v210: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:12:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:47 smithi204 ceph-mon[35013]: pgmap v210: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-12T03:12:49.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:49 smithi181 ceph-mon[35320]: pgmap v211: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:12:50.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:49 smithi057 ceph-mon[30361]: pgmap v211: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:12:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:49 smithi204 ceph-mon[35013]: pgmap v211: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-09-12T03:12:52.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:51 smithi057 ceph-mon[30361]: pgmap v212: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:51 smithi204 ceph-mon[35013]: pgmap v212: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:52.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:51 smithi181 ceph-mon[35320]: pgmap v212: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:54.005 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:53 smithi181 ceph-mon[35320]: pgmap v213: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:12:54.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:53 smithi057 ceph-mon[30361]: pgmap v213: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:12:54.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:53 smithi204 ceph-mon[35013]: pgmap v213: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:12:56.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:55 smithi057 ceph-mon[30361]: pgmap v214: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:55 smithi204 ceph-mon[35013]: pgmap v214: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:55 smithi181 ceph-mon[35320]: pgmap v214: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:57 smithi057 ceph-mon[30361]: pgmap v215: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:57 smithi204 ceph-mon[35013]: pgmap v215: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:57 smithi181 ceph-mon[35320]: pgmap v215: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.794+0000 7f0d87e00640 1 -- 172.21.15.57:0/2009332878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 msgr2=0x7f0d80104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.794+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2009332878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d80104310 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d7402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 -- 172.21.15.57:0/2009332878 shutdown_connections 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2009332878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 0x7f0d80110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2009332878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d801048e0 0x7f0d80108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2009332878 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d80104310 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 -- 172.21.15.57:0/2009332878 >> 172.21.15.57:0/2009332878 conn(0x7f0d800ff980 msgr2=0x7f0d80101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.796 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 -- 172.21.15.57:0/2009332878 shutdown_connections 2024-09-12T03:12:59.797 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.795+0000 7f0d87e00640 1 -- 172.21.15.57:0/2009332878 wait complete. 2024-09-12T03:12:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 Processor -- start 2024-09-12T03:12:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 -- start start 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d8019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 0x7f0d8019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801140f0 con 0x7f0d80103f30 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80113f70 con 0x7f0d80109080 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d87e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80114270 con 0x7f0d8019ee20 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d85400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d8019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:37558/0 (socket says 172.21.15.57:37558) 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d85e00640 1 -- 172.21.15.57:0/2886612227 learned_addr learned my addr 172.21.15.57:0/2886612227 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d84a00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 0x7f0d8019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.796+0000 7f0d85400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d8019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39946/0 (socket says 172.21.15.57:39946) 2024-09-12T03:12:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 -- 172.21.15.57:0/2886612227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 msgr2=0x7f0d8019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 0x7f0d8019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 -- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 msgr2=0x7f0d8019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d8019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 -- 172.21.15.57:0/2886612227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8019f910 con 0x7f0d8019ee20 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009fd0 tx=0x7f0d7800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.797+0000 7f0d7e000640 1 -- 172.21.15.57:0/2886612227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78017070 con 0x7f0d8019ee20 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.798+0000 7f0d7e000640 1 -- 172.21.15.57:0/2886612227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d78003e50 con 0x7f0d8019ee20 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.798+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8019fc00 con 0x7f0d8019ee20 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.798+0000 7f0d7e000640 1 -- 172.21.15.57:0/2886612227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780106c0 con 0x7f0d8019ee20 2024-09-12T03:12:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.798+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a73a0 con 0x7f0d8019ee20 2024-09-12T03:12:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 Processor -- start 2024-09-12T03:12:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d7e000640 1 -- 172.21.15.57:0/2886612227 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f0d78003970 con 0x7f0d8019ee20 2024-09-12T03:12:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 -- start start 2024-09-12T03:12:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562e0c12a100 con 0x562e0c1c4b00 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562e0c0d21b0 con 0x562e0c1c41e0 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.801+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562e0c1260c0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39948/0 (socket says 172.21.15.57:39948) 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 -- 172.21.15.57:0/990216861 learned_addr learned my addr 172.21.15.57:0/990216861 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d84a00640 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 -- 172.21.15.57:0/990216861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 msgr2=0x562e0c1d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 -- 172.21.15.57:0/990216861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 msgr2=0x562e0c1c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d85400640 1 -- 172.21.15.57:0/990216861 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0be3d210 con 0x562e0c1c4b00 2024-09-12T03:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.802+0000 7f0d7e000640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 0x7f0d680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d85400640 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67db7de9570ca32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d7e000640 1 -- 172.21.15.57:0/2886612227 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f0d780cf580 con 0x7f0d8019ee20 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d84a00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 0x7f0d680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d72600640 1 -- 172.21.15.57:0/990216861 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74035040 con 0x562e0c1c4b00 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d72600640 1 -- 172.21.15.57:0/990216861 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74005520 con 0x562e0c1c4b00 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.803+0000 7f0d72600640 1 -- 172.21.15.57:0/990216861 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d740037f0 con 0x562e0c1c4b00 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/990216861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 msgr2=0x562e0c1cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d84a00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 0x7f0d680a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562e0c1c1ad0 tx=0x7f0d6c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d74004040 comp rx=0 tx=0).stop 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/990216861 shutdown_connections 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cee60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/990216861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/990216861 >> 172.21.15.57:0/990216861 conn(0x562e0c1c0750 msgr2=0x562e0c1c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/990216861 shutdown_connections 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/990216861 wait complete. 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.804+0000 7f0d8ae1e740 1 -- start start 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 Processor -- start 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x562e0c1d1f70 con 0x562e0c1c4b00 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x562e0c0b3090 con 0x562e0c1c41e0 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562e0c1d20f0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d8ae1e740 10 client.0 _ll_register_callbacks cb 0x562e0c107f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d85400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d84a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39956/0 (socket says 172.21.15.57:39956) 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d84a00640 1 -- 172.21.15.57:0/1595164199 learned_addr learned my addr 172.21.15.57:0/1595164199 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.805+0000 7f0d85e00640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 -- 172.21.15.57:0/1595164199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 msgr2=0x562e0c1ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 -- 172.21.15.57:0/1595164199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 msgr2=0x562e0c1cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 -- 172.21.15.57:0/1595164199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0bf09620 con 0x562e0c1c41e0 2024-09-12T03:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d85400640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1cc310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d74031550 tx=0x7f0d7402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45716cd11db3a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:12:59.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.806+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74053060 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d74005520 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f0d74004e10 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d74035050 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 10 client.25739 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e0c1ddba0 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74005520 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d74031920 con 0x562e0c1c41e0 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 10 client.25739 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 20 client.25739 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c1deef0 0x562e0c1e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 4 client.25739 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.807+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e0c137830 con 0x562e0c1deef0 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.808+0000 7f0d84a00640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c1deef0 0x562e0c1e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.808+0000 7f0d84a00640 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c1deef0 0x562e0c1e12e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c30698dec2ee8d server_cookie=2ffa475df0de6faf in_seq=0 out_seq=0 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.808+0000 7f0d66a00640 10 client.25739 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.809+0000 7f0d66a00640 1 -- 172.21.15.57:0/1595164199 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e0c137830 con 0x562e0c1deef0 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.809+0000 7f0d66a00640 10 client.25739 handle_command_reply: tid=0 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.809+0000 7f0d8ae1e740 1 client.25739 shutdown 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 msgr2=0x562e0c1cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1cc310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d74031550 tx=0x7f0d7402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 shutdown_connections 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c1deef0 0x562e0c1e12e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1cf3a0 0x562e0c1ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1c4b00 0x562e0c1cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1595164199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c41e0 0x562e0c1cc310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d66a00640 0 client.25739 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 >> 172.21.15.57:0/1595164199 conn(0x562e0c1c0750 msgr2=0x562e0c1c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 shutdown_connections 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1595164199 wait complete. 2024-09-12T03:12:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.810+0000 7f0d8ae1e740 20 client.25739 trim_cache size 0 max 16384 2024-09-12T03:12:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.825+0000 7f0d8ae1e740 1 Processor -- start 2024-09-12T03:12:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.825+0000 7f0d8ae1e740 1 -- start start 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b3bc0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b1490 con 0x562e0c1c4b00 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801af300 con 0x562e0c1c41e0 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39962/0 (socket says 172.21.15.57:39962) 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 -- 172.21.15.57:0/2111671962 learned_addr learned my addr 172.21.15.57:0/2111671962 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85400640 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d84a00640 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 -- 172.21.15.57:0/2111671962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 msgr2=0x562e0c1ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 -- 172.21.15.57:0/2111671962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 msgr2=0x562e0c1e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.826+0000 7f0d85e00640 1 -- 172.21.15.57:0/2111671962 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0c1c3f60 con 0x562e0c1cf3a0 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d85e00640 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0d7809b3e0 tx=0x7f0d7809b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7d847ed518bf72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d71c00640 1 -- 172.21.15.57:0/2111671962 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78020040 con 0x562e0c1cf3a0 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d71c00640 1 -- 172.21.15.57:0/2111671962 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d780a42a0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d71c00640 1 -- 172.21.15.57:0/2111671962 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78097d50 con 0x562e0c1cf3a0 2024-09-12T03:12:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/2111671962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 msgr2=0x562e0c1e6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.827+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0d7809b3e0 tx=0x7f0d7809b040 comp rx=0 tx=0).stop 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/2111671962 shutdown_connections 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1e6e50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/2111671962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/2111671962 >> 172.21.15.57:0/2111671962 conn(0x562e0c1c0750 msgr2=0x562e0c1cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/2111671962 shutdown_connections 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/2111671962 wait complete. 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 -- start start 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.828+0000 7f0d8ae1e740 1 Processor -- start 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1d7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801af950 con 0x562e0c1cf3a0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b0dc0 con 0x562e0c1c4b00 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0d801b0020 con 0x562e0c1c41e0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d8ae1e740 10 client.0 _ll_register_callbacks cb 0x7f0d6c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39976/0 (socket says 172.21.15.57:39976) 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 -- 172.21.15.57:0/1125209748 learned_addr learned my addr 172.21.15.57:0/1125209748 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d84a00640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85400640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1d7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 -- 172.21.15.57:0/1125209748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 msgr2=0x562e0c1d7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1d7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 -- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 msgr2=0x562e0c1c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d85e00640 1 -- 172.21.15.57:0/1125209748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0c1c9ea0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.829+0000 7f0d84a00640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d85e00640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0d780a3d70 tx=0x7f0d780044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fe5bfbfc9c44cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d66a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d780b4020 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9554+0+0 (secure 0 0 0) 0x7f0d780ccae0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d780a42a0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d78020040 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d8ae1e740 10 client.15597 fetch_fsmap learned FSMap version 30 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e0c249990 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.830+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d780a42a0 con 0x562e0c1cf3a0 2024-09-12T03:12:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d780afb50 con 0x562e0c1cf3a0 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 10 client.15597 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 20 client.15597 populate_metadata read hostname 'smithi057' 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c249fa0 0x562e0c24c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 4 client.15597 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e0c131b80 con 0x562e0c249fa0 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.831+0000 7f0d85400640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c249fa0 0x562e0c24c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.832+0000 7f0d85400640 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c249fa0 0x562e0c24c3c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616b2ee481fc27b9 server_cookie=c4dbbfba5aa4f42e in_seq=0 out_seq=0 2024-09-12T03:12:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.832+0000 7f0d66a00640 10 client.15597 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d66a00640 1 -- 172.21.15.57:0/1125209748 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e0c131b80 con 0x562e0c249fa0 2024-09-12T03:12:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d66a00640 10 client.15597 handle_command_reply: tid=0 2024-09-12T03:12:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 client.15597 shutdown 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 msgr2=0x562e0c1c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0d780a3d70 tx=0x7f0d780044b0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 shutdown_connections 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x562e0c249fa0 0x562e0c24c3c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x562e0c1cf3a0 0x562e0c1c9670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x562e0c1c4b00 0x562e0c1c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d66a00640 0 client.15597 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 --2- 172.21.15.57:0/1125209748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562e0c1c41e0 0x562e0c1d7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 >> 172.21.15.57:0/1125209748 conn(0x562e0c1c0750 msgr2=0x562e0c1ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 shutdown_connections 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.836+0000 7f0d8ae1e740 1 -- 172.21.15.57:0/1125209748 wait complete. 2024-09-12T03:12:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d8ae1e740 20 client.15597 trim_cache size 0 max 16384 2024-09-12T03:12:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:12:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:12:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:12:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:12:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 msgr2=0x7f0d680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 0x7f0d680a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x562e0c1c1ad0 tx=0x7f0d6c005f90 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 msgr2=0x7f0d8019f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009fd0 tx=0x7f0d7800b260 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 shutdown_connections 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d8019ee20 0x7f0d8019f2d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d80109080 0x7f0d8019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f0d680a2da0 0x7f0d680a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 --2- 172.21.15.57:0/2886612227 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f0d80103f30 0x7f0d8019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 >> 172.21.15.57:0/2886612227 conn(0x7f0d800ff980 msgr2=0x7f0d80100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 shutdown_connections 2024-09-12T03:12:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:12:59.837+0000 7f0d87e00640 1 -- 172.21.15.57:0/2886612227 wait complete. 2024-09-12T03:13:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:12:59 smithi057 ceph-mon[30361]: pgmap v216: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:13:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:12:59 smithi204 ceph-mon[35013]: pgmap v216: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:13:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:12:59 smithi181 ceph-mon[35320]: pgmap v216: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-12T03:13:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:01 smithi057 ceph-mon[30361]: pgmap v217: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:01 smithi204 ceph-mon[35013]: pgmap v217: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:01 smithi181 ceph-mon[35320]: pgmap v217: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:03 smithi057 ceph-mon[30361]: pgmap v218: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:13:04.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:03 smithi204 ceph-mon[35013]: pgmap v218: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:13:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:03 smithi181 ceph-mon[35320]: pgmap v218: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:13:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:05 smithi057 ceph-mon[30361]: pgmap v219: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:13:06.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:05 smithi204 ceph-mon[35013]: pgmap v219: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:13:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:05 smithi181 ceph-mon[35320]: pgmap v219: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:13:08.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:07 smithi057 ceph-mon[30361]: pgmap v220: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:13:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:07 smithi204 ceph-mon[35013]: pgmap v220: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:13:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:07 smithi181 ceph-mon[35320]: pgmap v220: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:13:09.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:08 smithi057 ceph-mon[30361]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:13:09.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:08 smithi204 ceph-mon[35013]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:13:09.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:08 smithi181 ceph-mon[35320]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T03:13:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:09 smithi057 ceph-mon[30361]: pgmap v222: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:10.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:09 smithi204 ceph-mon[35013]: pgmap v222: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:10.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:09 smithi181 ceph-mon[35320]: pgmap v222: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:12.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:11 smithi057 ceph-mon[30361]: pgmap v223: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:11 smithi204 ceph-mon[35013]: pgmap v223: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:12.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:11 smithi181 ceph-mon[35320]: pgmap v223: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:13:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 -- 172.21.15.57:0/32143777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a810e380 msgr2=0x7fa9a8110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 --2- 172.21.15.57:0/32143777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a810e380 0x7fa9a8110770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009900 tx=0x7fa9a002a480 comp rx=0 tx=0).stop 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 -- 172.21.15.57:0/32143777 shutdown_connections 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 --2- 172.21.15.57:0/32143777 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a810e380 0x7fa9a8110770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 --2- 172.21.15.57:0/32143777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a8074ec0 0x7fa9a810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 --2- 172.21.15.57:0/32143777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a8074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 -- 172.21.15.57:0/32143777 >> 172.21.15.57:0/32143777 conn(0x7fa9a80fd7e0 msgr2=0x7fa9a80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 -- 172.21.15.57:0/32143777 shutdown_connections 2024-09-12T03:13:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.965+0000 7fa9ad000640 1 -- 172.21.15.57:0/32143777 wait complete. 2024-09-12T03:13:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 Processor -- start 2024-09-12T03:13:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 -- start start 2024-09-12T03:13:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 0x7fa9a819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 0x7fa9a819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8114090 con 0x7fa9a8074ec0 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8113f10 con 0x7fa9a810e380 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9ad000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8114210 con 0x7fa9a80745a0 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.966+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:39250/0 (socket says 172.21.15.57:39250) 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3671841277 learned_addr learned my addr 172.21.15.57:0/3671841277 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5400640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 0x7fa9a819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a6800640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 0x7fa9a819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3671841277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 msgr2=0x7fa9a819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 0x7fa9a819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 msgr2=0x7fa9a819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 0x7fa9a819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3671841277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a819f590 con 0x7fa9a80745a0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a6800640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 0x7fa9a819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5400640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 0x7fa9a819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.967+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa99800eb90 tx=0x7fa99800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.968+0000 7fa99ea00640 1 -- 172.21.15.57:0/3671841277 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004d20 con 0x7fa9a80745a0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.968+0000 7fa99ea00640 1 -- 172.21.15.57:0/3671841277 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99800ee90 con 0x7fa9a80745a0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.968+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a819f880 con 0x7fa9a80745a0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.968+0000 7fa99ea00640 1 -- 172.21.15.57:0/3671841277 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998010830 con 0x7fa9a80745a0 2024-09-12T03:13:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.968+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a81a7110 con 0x7fa9a80745a0 2024-09-12T03:13:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 Processor -- start 2024-09-12T03:13:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa99ea00640 1 -- 172.21.15.57:0/3671841277 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fa998005020 con 0x7fa9a80745a0 2024-09-12T03:13:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 -- start start 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b63790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0abc100 con 0x55cbd0b613a0 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0a641b0 con 0x55cbd0b56b00 2024-09-12T03:13:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.971+0000 7fa9afe92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0ab80c0 con 0x55cbd0b561e0 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:39262/0 (socket says 172.21.15.57:39262) 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a6800640 1 -- 172.21.15.57:0/3035594518 learned_addr learned my addr 172.21.15.57:0/3035594518 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5400640 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b63790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3035594518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 msgr2=0x55cbd0b565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3035594518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 msgr2=0x55cbd0b63790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b63790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.972+0000 7fa9a5e00640 1 -- 172.21.15.57:0/3035594518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd07cf210 con 0x55cbd0b56b00 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa99ea00640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 0x7fa98c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b60e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd230 tx=0x7fa9980d80e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3253e0f9f85fad30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa99ea00640 1 -- 172.21.15.57:0/3671841277 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9642+0+0 (secure 0 0 0) 0x7fa9980cff40 con 0x7fa9a80745a0 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa9a6800640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 0x7fa98c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa997400640 1 -- 172.21.15.57:0/3035594518 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980982a0 con 0x55cbd0b56b00 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa997400640 1 -- 172.21.15.57:0/3035594518 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99809c700 con 0x55cbd0b56b00 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.973+0000 7fa997400640 1 -- 172.21.15.57:0/3035594518 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99809b960 con 0x55cbd0b56b00 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- 172.21.15.57:0/3035594518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 msgr2=0x55cbd0b60e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b60e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd230 tx=0x7fa9980d80e0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9a6800640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 0x7fa98c0a52e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0005940 tx=0x7fa9a00058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- 172.21.15.57:0/3035594518 shutdown_connections 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b63790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b60e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 --2- 172.21.15.57:0/3035594518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- 172.21.15.57:0/3035594518 >> 172.21.15.57:0/3035594518 conn(0x55cbd0b52750 msgr2=0x55cbd0b52b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- 172.21.15.57:0/3035594518 shutdown_connections 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- 172.21.15.57:0/3035594518 wait complete. 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.974+0000 7fa9afe92740 1 -- start start 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 Processor -- start 2024-09-12T03:13:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b5e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0b63f70 con 0x55cbd0b56b00 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0a45090 con 0x55cbd0b613a0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cbd0b640f0 con 0x55cbd0b561e0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9afe92740 10 client.0 _ll_register_callbacks cb 0x55cbd0a99f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9a5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47076/0 (socket says 172.21.15.57:47076) 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9a5400640 1 -- 172.21.15.57:0/4005440432 learned_addr learned my addr 172.21.15.57:0/4005440432 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9a6800640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.975+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b5e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 -- 172.21.15.57:0/4005440432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 msgr2=0x55cbd0b5e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b5e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 -- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 msgr2=0x55cbd0b5ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 -- 172.21.15.57:0/4005440432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd089b620 con 0x55cbd0b56b00 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b5e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9a5400640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa990012d50 tx=0x7fa99000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec843c240923a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa98be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990012110 con 0x55cbd0b56b00 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa99809c700 con 0x55cbd0b56b00 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.976+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9642+0+0 (secure 0 0 0) 0x7fa990004050 con 0x55cbd0b56b00 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9900122b0 con 0x55cbd0b56b00 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 10 client.15609 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbd0b6fc50 con 0x55cbd0b56b00 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99809c700 con 0x55cbd0b56b00 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa990010430 con 0x55cbd0b56b00 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 20 client.15609 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.977+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0b710c0 0x55cbd0b73510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.978+0000 7fa9afe92740 4 client.15609 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.978+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd0ac9830 con 0x55cbd0b710c0 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.978+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0b710c0 0x55cbd0b73510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.978+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0b710c0 0x55cbd0b73510 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84048d3e7d9a2287 server_cookie=c96228b1f1363883 in_seq=0 out_seq=0 2024-09-12T03:13:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.978+0000 7fa98be00640 10 client.15609 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa98be00640 1 -- 172.21.15.57:0/4005440432 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbd0ac9830 con 0x55cbd0b710c0 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa98be00640 10 client.15609 handle_command_reply: tid=0 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 client.15609 shutdown 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 msgr2=0x55cbd0b5e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa990012d50 tx=0x7fa99000f790 comp rx=0 tx=0).stop 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 shutdown_connections 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0b710c0 0x55cbd0b73510 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5e850 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.979+0000 7fa9afe92740 1 --2- 172.21.15.57:0/4005440432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b5e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.980+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 >> 172.21.15.57:0/4005440432 conn(0x55cbd0b52750 msgr2=0x55cbd0b58460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.980+0000 7fa98be00640 0 client.15609 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.980+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 shutdown_connections 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.980+0000 7fa9afe92740 1 -- 172.21.15.57:0/4005440432 wait complete. 2024-09-12T03:13:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.980+0000 7fa9afe92740 20 client.15609 trim_cache size 0 max 16384 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.995+0000 7fa9afe92740 1 Processor -- start 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.995+0000 7fa9afe92740 1 -- start start 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.995+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.995+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 0x55cbd0b78e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.995+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b79360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9afe92740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b3d30 con 0x55cbd0b561e0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9afe92740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b1600 con 0x55cbd0b613a0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9afe92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81af470 con 0x55cbd0b56b00 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47082/0 (socket says 172.21.15.57:47082) 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1809715682 learned_addr learned my addr 172.21.15.57:0/1809715682 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5400640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 0x55cbd0b78e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a6800640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b79360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1809715682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 msgr2=0x55cbd0b78e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 0x55cbd0b78e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1809715682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 msgr2=0x55cbd0b79360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b79360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1809715682 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd0b56010 con 0x55cbd0b561e0 2024-09-12T03:13:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5400640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 0x55cbd0b78e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.996+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd830 tx=0x7fa9980bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84ca04e5c30fa5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa996a00640 1 -- 172.21.15.57:0/1809715682 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99809bb40 con 0x55cbd0b561e0 2024-09-12T03:13:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa996a00640 1 -- 172.21.15.57:0/1809715682 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9980d3860 con 0x55cbd0b561e0 2024-09-12T03:13:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa996a00640 1 -- 172.21.15.57:0/1809715682 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99809bdd0 con 0x55cbd0b561e0 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 -- 172.21.15.57:0/1809715682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 msgr2=0x55cbd0b60170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cd830 tx=0x7fa9980bc040 comp rx=0 tx=0).stop 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 -- 172.21.15.57:0/1809715682 shutdown_connections 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b613a0 0x55cbd0b79360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b56b00 0x55cbd0b78e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1809715682 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b561e0 0x55cbd0b60170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 -- 172.21.15.57:0/1809715682 >> 172.21.15.57:0/1809715682 conn(0x55cbd0b52750 msgr2=0x55cbd0b5fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.997+0000 7fa9afe92740 1 -- 172.21.15.57:0/1809715682 shutdown_connections 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 -- 172.21.15.57:0/1809715682 wait complete. 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 -- start start 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 Processor -- start 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b69860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81afac0 con 0x55cbd0b613a0 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b0f30 con 0x55cbd0b56b00 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.998+0000 7fa9afe92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a81b0190 con 0x55cbd0b561e0 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9afe92740 10 client.0 _ll_register_callbacks cb 0x7fa990009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b69860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:47092/0 (socket says 172.21.15.57:47092) 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a6800640 1 -- 172.21.15.57:0/1004600697 learned_addr learned my addr 172.21.15.57:0/1004600697 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5400640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 msgr2=0x55cbd0b5ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1004600697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 msgr2=0x55cbd0b5b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 -- 172.21.15.57:0/1004600697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd0b5b8e0 con 0x55cbd0b561e0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa9a5e00640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b69860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa99800cba0 tx=0x7fa99809c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b28da60418fe580 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:12.999+0000 7fa98be00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9980b4070 con 0x55cbd0b561e0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9642+0+0 (secure 0 0 0) 0x7fa9980d3350 con 0x55cbd0b561e0 2024-09-12T03:13:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9980d3860 con 0x55cbd0b561e0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa99809bdd0 con 0x55cbd0b561e0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa9afe92740 10 client.25783 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbd0bdbb30 con 0x55cbd0b561e0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.000+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9980d3860 con 0x55cbd0b561e0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa9980cd4b0 con 0x55cbd0b561e0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 10 client.25783 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 20 client.25783 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0bdbdb0 0x55cbd0bde1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 4 client.25783 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbd0ac3b80 con 0x55cbd0bdbdb0 2024-09-12T03:13:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9a5400640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0bdbdb0 0x55cbd0bde1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa9a5400640 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0bdbdb0 0x55cbd0bde1a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60f95897a094dc6 server_cookie=ba8b0feb955ae91c in_seq=0 out_seq=0 2024-09-12T03:13:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.001+0000 7fa98be00640 10 client.25783 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa98be00640 1 -- 172.21.15.57:0/1004600697 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbd0ac3b80 con 0x55cbd0bdbdb0 2024-09-12T03:13:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa98be00640 10 client.25783 handle_command_reply: tid=0 2024-09-12T03:13:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 client.25783 shutdown 2024-09-12T03:13:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 msgr2=0x55cbd0b69860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.009 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b69860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa99800cba0 tx=0x7fa99809c0c0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 shutdown_connections 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55cbd0bdbdb0 0x55cbd0bde1a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55cbd0b613a0 0x55cbd0b5b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55cbd0b56b00 0x55cbd0b5ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 --2- 172.21.15.57:0/1004600697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbd0b561e0 0x55cbd0b69860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 >> 172.21.15.57:0/1004600697 conn(0x55cbd0b52750 msgr2=0x55cbd0b5ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.008+0000 7fa98be00640 0 client.25783 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 shutdown_connections 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9afe92740 1 -- 172.21.15.57:0/1004600697 wait complete. 2024-09-12T03:13:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9afe92740 20 client.25783 trim_cache size 0 max 16384 2024-09-12T03:13:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:13:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:13:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:13:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:13:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 msgr2=0x7fa98c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 0x7fa98c0a52e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0005940 tx=0x7fa9a00058d0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 msgr2=0x7fa9a819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa99800eb90 tx=0x7fa99800c6a0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 shutdown_connections 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a810e380 0x7fa9a819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa98c0a2e20 0x7fa98c0a52e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa9a8074ec0 0x7fa9a819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.009+0000 7fa9ad000640 1 --2- 172.21.15.57:0/3671841277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9a80745a0 0x7fa9a819a5a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.010+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 >> 172.21.15.57:0/3671841277 conn(0x7fa9a80fd7e0 msgr2=0x7fa9a80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.010+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 shutdown_connections 2024-09-12T03:13:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:13.010+0000 7fa9ad000640 1 -- 172.21.15.57:0/3671841277 wait complete. 2024-09-12T03:13:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:13 smithi057 ceph-mon[30361]: pgmap v224: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:13 smithi204 ceph-mon[35013]: pgmap v224: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:13 smithi181 ceph-mon[35320]: pgmap v224: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:16.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:15 smithi057 ceph-mon[30361]: pgmap v225: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:15 smithi204 ceph-mon[35013]: pgmap v225: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:16.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:15 smithi181 ceph-mon[35320]: pgmap v225: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-09-12T03:13:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:17 smithi057 ceph-mon[30361]: pgmap v226: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:13:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:17 smithi057 ceph-mon[30361]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:13:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:17 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:17 smithi204 ceph-mon[35013]: pgmap v226: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:13:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:17 smithi204 ceph-mon[35013]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:13:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:17 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:17 smithi181 ceph-mon[35320]: pgmap v226: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-09-12T03:13:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:17 smithi181 ceph-mon[35320]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T03:13:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:17 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:18.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:18 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:18.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:18 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:18.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:18 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:13:18.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:18 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:18.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:18 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:18.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:18 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:13:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:18 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:18 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:18 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:13:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:19 smithi057 ceph-mon[30361]: pgmap v228: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:19 smithi204 ceph-mon[35013]: pgmap v228: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:19 smithi181 ceph-mon[35320]: pgmap v228: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:21 smithi057 ceph-mon[30361]: pgmap v229: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:21 smithi204 ceph-mon[35013]: pgmap v229: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:22.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:21 smithi181 ceph-mon[35320]: pgmap v229: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 8 op/s 2024-09-12T03:13:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:23 smithi057 ceph-mon[30361]: pgmap v230: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:24.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:23 smithi204 ceph-mon[35013]: pgmap v230: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:23 smithi181 ceph-mon[35320]: pgmap v230: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:26.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:25 smithi057 ceph-mon[30361]: pgmap v231: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 -- 172.21.15.57:0/413031615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8441018b0 msgr2=0x7fc844101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 --2- 172.21.15.57:0/413031615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8441018b0 0x7fc844101d10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc830009890 tx=0x7fc83002a380 comp rx=0 tx=0).stop 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 -- 172.21.15.57:0/413031615 shutdown_connections 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 --2- 172.21.15.57:0/413031615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc844102250 0x7fc84410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 --2- 172.21.15.57:0/413031615 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc8441018b0 0x7fc844101d10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 --2- 172.21.15.57:0/413031615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc844107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.138+0000 7fc843e00640 1 -- 172.21.15.57:0/413031615 >> 172.21.15.57:0/413031615 conn(0x7fc8440fd5d0 msgr2=0x7fc8440ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.139+0000 7fc843e00640 1 -- 172.21.15.57:0/413031615 shutdown_connections 2024-09-12T03:13:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.139+0000 7fc843e00640 1 -- 172.21.15.57:0/413031615 wait complete. 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.139+0000 7fc843e00640 1 Processor -- start 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 -- start start 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8441018b0 0x7fc84419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc84419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc844111d00 con 0x7fc844102250 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc844111b80 con 0x7fc8441078c0 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc844111e80 con 0x7fc8441018b0 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc842000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc842000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45326/0 (socket says 172.21.15.57:45326) 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc843400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc84419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc842000640 1 -- 172.21.15.57:0/1564145531 learned_addr learned my addr 172.21.15.57:0/1564145531 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.140+0000 7fc842a00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8441018b0 0x7fc84419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 -- 172.21.15.57:0/1564145531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8441018b0 msgr2=0x7fc84419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8441018b0 0x7fc84419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 -- 172.21.15.57:0/1564145531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 msgr2=0x7fc84419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc84419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 -- 172.21.15.57:0/1564145531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84419f420 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc843400640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc84419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc842000640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc830009860 tx=0x7fc830033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc83b400640 1 -- 172.21.15.57:0/1564145531 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83003f020 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc83b400640 1 -- 172.21.15.57:0/1564145531 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc830032770 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84419f6b0 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.141+0000 7fc83b400640 1 -- 172.21.15.57:0/1564145531 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830037040 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.142+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8441a6fa0 con 0x7fc844102250 2024-09-12T03:13:26.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:25 smithi204 ceph-mon[35013]: pgmap v231: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc83b400640 1 -- 172.21.15.57:0/1564145531 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fc830042020 con 0x7fc844102250 2024-09-12T03:13:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 Processor -- start 2024-09-12T03:13:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 -- start start 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 0x561ae9db3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9db3570 0x561ae9db5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561ae9d0e100 con 0x561ae9da8cd0 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561ae9cb61b0 con 0x561ae9db3570 2024-09-12T03:13:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.145+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561ae9d0a0c0 con 0x561ae9da83b0 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc842a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 0x561ae9db3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:41954/0 (socket says 172.21.15.57:41954) 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 -- 172.21.15.57:0/2153404205 learned_addr learned my addr 172.21.15.57:0/2153404205 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc842000640 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9db3570 0x561ae9db5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 -- 172.21.15.57:0/2153404205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9db3570 msgr2=0x561ae9db5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9db3570 0x561ae9db5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 -- 172.21.15.57:0/2153404205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 msgr2=0x561ae9db3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 0x561ae9db3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc83b400640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 0x7fc8280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc843400640 1 -- 172.21.15.57:0/2153404205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ae9a21210 con 0x561ae9da83b0 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc842a00640 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 0x561ae9db3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.146+0000 7fc83b400640 1 -- 172.21.15.57:0/1564145531 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fc8300ef7f0 con 0x7fc844102250 2024-09-12T03:13:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.147+0000 7fc843400640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 0x7fc8280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.147+0000 7fc843400640 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0117e0 tx=0x7fc83c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5e019461fee702 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.147+0000 7fc82fe00640 1 -- 172.21.15.57:0/2153404205 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c010570 con 0x561ae9da83b0 2024-09-12T03:13:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.147+0000 7fc82fe00640 1 -- 172.21.15.57:0/2153404205 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83c010bf0 con 0x561ae9da83b0 2024-09-12T03:13:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.147+0000 7fc82fe00640 1 -- 172.21.15.57:0/2153404205 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c01caf0 con 0x561ae9da83b0 2024-09-12T03:13:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 -- 172.21.15.57:0/2153404205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 msgr2=0x561ae9da8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0117e0 tx=0x7fc83c011ca0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc843400640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 0x7fc8280a51b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc83c011490 tx=0x7fc83c021000 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 -- 172.21.15.57:0/2153404205 shutdown_connections 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9db3570 0x561ae9db5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da8cd0 0x561ae9db3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/2153404205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9da8790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 -- 172.21.15.57:0/2153404205 >> 172.21.15.57:0/2153404205 conn(0x561ae9da4920 msgr2=0x561ae9da4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 -- 172.21.15.57:0/2153404205 shutdown_connections 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.148+0000 7fc84c8c7740 1 -- 172.21.15.57:0/2153404205 wait complete. 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 -- start start 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 Processor -- start 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9db0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9db3570 0x561ae9db10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x561ae9db5f90 con 0x561ae9da83b0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x561ae9c97090 con 0x561ae9da8cd0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561ae9db6110 con 0x561ae9db3570 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.149+0000 7fc84c8c7740 10 client.0 _ll_register_callbacks cb 0x561ae9cebf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc843400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9db3570 0x561ae9db10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45356/0 (socket says 172.21.15.57:45356) 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 -- 172.21.15.57:0/3915948973 learned_addr learned my addr 172.21.15.57:0/3915948973 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842000640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9db0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 -- 172.21.15.57:0/3915948973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9db3570 msgr2=0x561ae9db10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9db3570 0x561ae9db10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 -- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 msgr2=0x561ae9db0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9db0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 -- 172.21.15.57:0/3915948973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ae9aed620 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc842a00640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc834011740 tx=0x7fc834011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf769d580ad096ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.150+0000 7fc82cc00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83400fb30 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fc83400fcd0 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc83c010bf0 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc834027050 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc84c8c7740 10 client.15627 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ae9dc1ed0 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83c010bf0 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.151+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc834022030 con 0x561ae9da83b0 2024-09-12T03:13:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 10 client.15627 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 20 client.15627 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9dc3390 0x561ae9dc57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 4 client.15627 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ae9d1b830 con 0x561ae9dc3390 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.152+0000 7fc842000640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9dc3390 0x561ae9dc57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.153+0000 7fc842000640 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9dc3390 0x561ae9dc57e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65006a1d686d333a server_cookie=fbde6afb9f0cf07 in_seq=0 out_seq=0 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.153+0000 7fc82cc00640 10 client.15627 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.153+0000 7fc82cc00640 1 -- 172.21.15.57:0/3915948973 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ae9d1b830 con 0x561ae9dc3390 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.153+0000 7fc82cc00640 10 client.15627 handle_command_reply: tid=0 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.153+0000 7fc84c8c7740 1 client.15627 shutdown 2024-09-12T03:13:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 msgr2=0x561ae9db0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc834011740 tx=0x7fc834011c00 comp rx=0 tx=0).stop 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 shutdown_connections 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9dc3390 0x561ae9dc57e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9db3570 0x561ae9db10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9db0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/3915948973 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9da83b0 0x561ae9db0630 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc82cc00640 0 client.15627 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 >> 172.21.15.57:0/3915948973 conn(0x561ae9da4920 msgr2=0x561ae9daa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 shutdown_connections 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 1 -- 172.21.15.57:0/3915948973 wait complete. 2024-09-12T03:13:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.154+0000 7fc84c8c7740 20 client.15627 trim_cache size 0 max 16384 2024-09-12T03:13:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.169+0000 7fc84c8c7740 1 Processor -- start 2024-09-12T03:13:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.169+0000 7fc84c8c7740 1 -- start start 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da8cd0 0x561ae9dcb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9da31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441b3a80 con 0x561ae9db3570 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441b1350 con 0x561ae9da83b0 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441af1c0 con 0x561ae9da8cd0 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc843400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9da31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51700/0 (socket says 172.21.15.57:51700) 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 -- 172.21.15.57:0/330148172 learned_addr learned my addr 172.21.15.57:0/330148172 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842000640 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da8cd0 0x561ae9dcb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 -- 172.21.15.57:0/330148172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da8cd0 msgr2=0x561ae9dcb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da8cd0 0x561ae9dcb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 -- 172.21.15.57:0/330148172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 msgr2=0x561ae9da31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9da31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.170+0000 7fc842a00640 1 -- 172.21.15.57:0/330148172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ae9dbf320 con 0x561ae9da83b0 2024-09-12T03:13:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc842a00640 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc834009340 tx=0x7fc83400f810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec2e87c81514778 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc82f400640 1 -- 172.21.15.57:0/330148172 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834011030 con 0x561ae9da83b0 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc82f400640 1 -- 172.21.15.57:0/330148172 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834011e50 con 0x561ae9da83b0 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc82f400640 1 -- 172.21.15.57:0/330148172 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8340052a0 con 0x561ae9da83b0 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc84c8c7740 1 -- 172.21.15.57:0/330148172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 msgr2=0x561ae9dcb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.171+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc834009340 tx=0x7fc83400f810 comp rx=0 tx=0).stop 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 -- 172.21.15.57:0/330148172 shutdown_connections 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9da31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da8cd0 0x561ae9dcb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/330148172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da83b0 0x561ae9dcb140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 -- 172.21.15.57:0/330148172 >> 172.21.15.57:0/330148172 conn(0x561ae9da4920 msgr2=0x561ae9dab890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 -- 172.21.15.57:0/330148172 shutdown_connections 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 -- 172.21.15.57:0/330148172 wait complete. 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 -- start start 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.172+0000 7fc84c8c7740 1 Processor -- start 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9dbbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9dace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441af810 con 0x561ae9db3570 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441b0c80 con 0x561ae9da8cd0 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8441afee0 con 0x561ae9da83b0 2024-09-12T03:13:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc84c8c7740 10 client.0 _ll_register_callbacks cb 0x7fc834009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc843400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9dace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9dbbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:41978/0 (socket says 172.21.15.57:41978) 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 -- 172.21.15.57:0/159722015 learned_addr learned my addr 172.21.15.57:0/159722015 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 -- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 msgr2=0x561ae9dbbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9dbbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 -- 172.21.15.57:0/159722015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 msgr2=0x561ae9dace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9dace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.173+0000 7fc842a00640 1 -- 172.21.15.57:0/159722015 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ae9dad490 con 0x561ae9da83b0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc842a00640 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834011260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9af9970b0472b4b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc82cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83403a060 con 0x561ae9da83b0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc834011e50 con 0x561ae9da83b0 2024-09-12T03:13:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fc834031040 con 0x561ae9da83b0 2024-09-12T03:13:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc83400fd40 con 0x561ae9da83b0 2024-09-12T03:13:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc84c8c7740 10 client.25801 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.174+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561ae9e2da80 con 0x561ae9da83b0 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834011e50 con 0x561ae9da83b0 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc834022040 con 0x561ae9da83b0 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 10 client.25801 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 20 client.25801 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9e2e030 0x561ae9e30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 4 client.25801 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ae9d15b80 con 0x561ae9e2e030 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.175+0000 7fc842000640 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9e2e030 0x561ae9e30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.176+0000 7fc842000640 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9e2e030 0x561ae9e30440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8432b3caa8aaeb2e server_cookie=e2abceeef0919715 in_seq=0 out_seq=0 2024-09-12T03:13:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.176+0000 7fc82cc00640 10 client.25801 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.184+0000 7fc82cc00640 1 -- 172.21.15.57:0/159722015 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ae9d15b80 con 0x561ae9e2e030 2024-09-12T03:13:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.184+0000 7fc82cc00640 10 client.25801 handle_command_reply: tid=0 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.184+0000 7fc84c8c7740 1 client.25801 shutdown 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.184+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 msgr2=0x561ae9dbb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.184+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834011260 comp rx=0 tx=0).stop 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 shutdown_connections 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x561ae9e2e030 0x561ae9e30440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x561ae9db3570 0x561ae9dace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x561ae9da8cd0 0x561ae9dbbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 --2- 172.21.15.57:0/159722015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ae9da83b0 0x561ae9dbb680 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc82cc00640 0 client.25801 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 >> 172.21.15.57:0/159722015 conn(0x561ae9da4920 msgr2=0x561ae9db05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 shutdown_connections 2024-09-12T03:13:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 1 -- 172.21.15.57:0/159722015 wait complete. 2024-09-12T03:13:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc84c8c7740 20 client.25801 trim_cache size 0 max 16384 2024-09-12T03:13:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:13:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:13:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:13:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 msgr2=0x7fc8280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 0x7fc8280a51b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc83c011490 tx=0x7fc83c021000 comp rx=0 tx=0).stop 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 msgr2=0x7fc84419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.185+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc830009860 tx=0x7fc830033d10 comp rx=0 tx=0).stop 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 shutdown_connections 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8441078c0 0x7fc84419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc8280a2cf0 0x7fc8280a51b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc844102250 0x7fc84419a970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 --2- 172.21.15.57:0/1564145531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8441018b0 0x7fc84419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 >> 172.21.15.57:0/1564145531 conn(0x7fc8440fd5d0 msgr2=0x7fc84410cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 shutdown_connections 2024-09-12T03:13:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:26.186+0000 7fc843e00640 1 -- 172.21.15.57:0/1564145531 wait complete. 2024-09-12T03:13:26.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:25 smithi181 ceph-mon[35320]: pgmap v231: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-09-12T03:13:28.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:27 smithi204 ceph-mon[35013]: pgmap v232: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-12T03:13:28.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:27 smithi181 ceph-mon[35320]: pgmap v232: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-12T03:13:28.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:27 smithi057 ceph-mon[30361]: pgmap v232: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.4 KiB/s wr, 7 op/s 2024-09-12T03:13:30.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:29 smithi204 ceph-mon[35013]: pgmap v233: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:13:30.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:29 smithi181 ceph-mon[35320]: pgmap v233: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:13:30.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:29 smithi057 ceph-mon[30361]: pgmap v233: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-12T03:13:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:31 smithi204 ceph-mon[35013]: pgmap v234: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-09-12T03:13:32.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:31 smithi181 ceph-mon[35320]: pgmap v234: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-09-12T03:13:32.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:31 smithi057 ceph-mon[30361]: pgmap v234: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-09-12T03:13:34.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:33 smithi204 ceph-mon[35013]: pgmap v235: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:34.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:33 smithi181 ceph-mon[35320]: pgmap v235: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:34.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:33 smithi057 ceph-mon[30361]: pgmap v235: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:36.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:35 smithi204 ceph-mon[35013]: pgmap v236: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T03:13:36.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:35 smithi181 ceph-mon[35320]: pgmap v236: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T03:13:36.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:35 smithi057 ceph-mon[30361]: pgmap v236: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-12T03:13:38.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:37 smithi181 ceph-mon[35320]: pgmap v237: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-12T03:13:38.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:37 smithi057 ceph-mon[30361]: pgmap v237: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-12T03:13:38.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:37 smithi204 ceph-mon[35013]: pgmap v237: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-09-12T03:13:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 -- 172.21.15.57:0/4153393092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624107b80 msgr2=0x7fe624107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 --2- 172.21.15.57:0/4153393092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624107b80 0x7fe624107f60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe618009890 tx=0x7fe61802a380 comp rx=0 tx=0).stop 2024-09-12T03:13:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 -- 172.21.15.57:0/4153393092 shutdown_connections 2024-09-12T03:13:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 --2- 172.21.15.57:0/4153393092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624102480 0x7fe62410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 --2- 172.21.15.57:0/4153393092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624101ae0 0x7fe624101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 --2- 172.21.15.57:0/4153393092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624107b80 0x7fe624107f60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 -- 172.21.15.57:0/4153393092 >> 172.21.15.57:0/4153393092 conn(0x7fe6240fd7e0 msgr2=0x7fe6240ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 -- 172.21.15.57:0/4153393092 shutdown_connections 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.312+0000 7fe62c000640 1 -- 172.21.15.57:0/4153393092 wait complete. 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 Processor -- start 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 -- start start 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 0x7fe62419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624107b80 0x7fe62419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe624111ef0 con 0x7fe624101ae0 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe624111d70 con 0x7fe624107b80 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.313+0000 7fe62c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe624112070 con 0x7fe624102480 2024-09-12T03:13:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46716/0 (socket says 172.21.15.57:46716) 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe629600640 1 -- 172.21.15.57:0/845249839 learned_addr learned my addr 172.21.15.57:0/845249839 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 0x7fe62419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe62a000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624107b80 0x7fe62419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 -- 172.21.15.57:0/845249839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624107b80 msgr2=0x7fe62419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624107b80 0x7fe62419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 -- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 msgr2=0x7fe62419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe628c00640 1 -- 172.21.15.57:0/845249839 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62419f580 con 0x7fe624102480 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.314+0000 7fe629600640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe628c00640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 0x7fe62419aad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe610005fd0 tx=0x7fe61000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe622000640 1 -- 172.21.15.57:0/845249839 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61000cd00 con 0x7fe624102480 2024-09-12T03:13:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe622000640 1 -- 172.21.15.57:0/845249839 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6100035f0 con 0x7fe624102480 2024-09-12T03:13:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe622000640 1 -- 172.21.15.57:0/845249839 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610010690 con 0x7fe624102480 2024-09-12T03:13:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62419f870 con 0x7fe624102480 2024-09-12T03:13:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.315+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241a7100 con 0x7fe624102480 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe62ee3c740 1 Processor -- start 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe622000640 1 -- 172.21.15.57:0/845249839 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fe6100037a0 con 0x7fe624102480 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe62ee3c740 1 -- start start 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.318+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e392ebd100 con 0x55e392f571e0 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e392e651b0 con 0x55e392f623a0 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e392eb90c0 con 0x55e392f57b00 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46718/0 (socket says 172.21.15.57:46718) 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 -- 172.21.15.57:0/2885930693 learned_addr learned my addr 172.21.15.57:0/2885930693 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe629600640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe628c00640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 -- 172.21.15.57:0/2885930693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 msgr2=0x55e392f61e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 -- 172.21.15.57:0/2885930693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 msgr2=0x55e392f64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe62a000640 1 -- 172.21.15.57:0/2885930693 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e392bd0210 con 0x55e392f571e0 2024-09-12T03:13:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.319+0000 7fe628c00640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe622000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 0x7fe60c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe62a000640 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009f50 tx=0x7fe61c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226e23e25033abf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe622000640 1 -- 172.21.15.57:0/845249839 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fe6100cfaa0 con 0x7fe624102480 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe629600640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 0x7fe60c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe616600640 1 -- 172.21.15.57:0/2885930693 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004d20 con 0x55e392f571e0 2024-09-12T03:13:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe616600640 1 -- 172.21.15.57:0/2885930693 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c004050 con 0x55e392f571e0 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.320+0000 7fe616600640 1 -- 172.21.15.57:0/2885930693 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c005220 con 0x55e392f571e0 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 -- 172.21.15.57:0/2885930693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 msgr2=0x55e392f575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009f50 tx=0x7fe61c0118d0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe629600640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 0x7fe60c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe618005e00 tx=0x7fe6180056d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 -- 172.21.15.57:0/2885930693 shutdown_connections 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/2885930693 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f575c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 -- 172.21.15.57:0/2885930693 >> 172.21.15.57:0/2885930693 conn(0x55e392f53750 msgr2=0x55e392f53b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 -- 172.21.15.57:0/2885930693 shutdown_connections 2024-09-12T03:13:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.321+0000 7fe62ee3c740 1 -- 172.21.15.57:0/2885930693 wait complete. 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 -- start start 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 Processor -- start 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f623a0 0x55e392f5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55e392f64f70 con 0x55e392f571e0 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55e392e46090 con 0x55e392f57b00 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e392f650f0 con 0x55e392f623a0 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe62ee3c740 10 client.0 _ll_register_callbacks cb 0x55e392e9af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.322+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46724/0 (socket says 172.21.15.57:46724) 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 -- 172.21.15.57:0/3876512893 learned_addr learned my addr 172.21.15.57:0/3876512893 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe628c00640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe62a000640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f623a0 0x55e392f5fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 -- 172.21.15.57:0/3876512893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f623a0 msgr2=0x55e392f5fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f623a0 0x55e392f5fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 -- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 msgr2=0x55e392f5f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f5f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 -- 172.21.15.57:0/3876512893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e392c9c620 con 0x55e392f571e0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe628c00640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f5f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe629600640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe618030430 tx=0x7fe61802a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4b3eec5af60225 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.323+0000 7fe60aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618043070 con 0x55e392f571e0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fe618003b80 con 0x55e392f571e0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61c004050 con 0x55e392f571e0 2024-09-12T03:13:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe618058050 con 0x55e392f571e0 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe62ee3c740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e392f70b50 con 0x55e392f571e0 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c004050 con 0x55e392f571e0 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.324+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe61803d450 con 0x55e392f571e0 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 20 client.15645 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392f72010 0x55e392f74400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 4 client.15645 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e392eca830 con 0x55e392f72010 2024-09-12T03:13:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.325+0000 7fe628c00640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392f72010 0x55e392f74400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.326+0000 7fe628c00640 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392f72010 0x55e392f74400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f38f5c12c541b9 server_cookie=ee607bbdc739c593 in_seq=0 out_seq=0 2024-09-12T03:13:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.326+0000 7fe60aa00640 10 client.15645 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.326+0000 7fe60aa00640 1 -- 172.21.15.57:0/3876512893 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e392eca830 con 0x55e392f72010 2024-09-12T03:13:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.326+0000 7fe60aa00640 10 client.15645 handle_command_reply: tid=0 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 client.15645 shutdown 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 msgr2=0x55e392f5f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe618030430 tx=0x7fe61802a380 comp rx=0 tx=0).stop 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 shutdown_connections 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392f72010 0x55e392f74400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f623a0 0x55e392f5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f5f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/3876512893 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f5f2d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe60aa00640 0 client.15645 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 >> 172.21.15.57:0/3876512893 conn(0x55e392f53750 msgr2=0x55e392f59460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 shutdown_connections 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 1 -- 172.21.15.57:0/3876512893 wait complete. 2024-09-12T03:13:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.327+0000 7fe62ee3c740 20 client.15645 trim_cache size 0 max 16384 2024-09-12T03:13:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.342+0000 7fe62ee3c740 1 Processor -- start 2024-09-12T03:13:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.342+0000 7fe62ee3c740 1 -- start start 2024-09-12T03:13:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241b3c00 con 0x55e392f571e0 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241b14d0 con 0x55e392f623a0 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241af340 con 0x55e392f57b00 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46736/0 (socket says 172.21.15.57:46736) 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 -- 172.21.15.57:0/1997140167 learned_addr learned my addr 172.21.15.57:0/1997140167 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe628c00640 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe62a000640 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 -- 172.21.15.57:0/1997140167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 msgr2=0x55e392f61130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 -- 172.21.15.57:0/1997140167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 msgr2=0x55e392f79d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f79d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.343+0000 7fe629600640 1 -- 172.21.15.57:0/1997140167 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e392f54800 con 0x55e392f571e0 2024-09-12T03:13:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.344+0000 7fe629600640 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe61803d840 tx=0x7fe618030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52794e906a460277 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.344+0000 7fe615c00640 1 -- 172.21.15.57:0/1997140167 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618060070 con 0x55e392f571e0 2024-09-12T03:13:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.344+0000 7fe615c00640 1 -- 172.21.15.57:0/1997140167 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61804f070 con 0x55e392f571e0 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.344+0000 7fe615c00640 1 -- 172.21.15.57:0/1997140167 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618043070 con 0x55e392f571e0 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- 172.21.15.57:0/1997140167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 msgr2=0x55e392f59c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe61803d840 tx=0x7fe618030340 comp rx=0 tx=0).stop 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- 172.21.15.57:0/1997140167 shutdown_connections 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f623a0 0x55e392f79d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f57b00 0x55e392f61130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/1997140167 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f571e0 0x55e392f59c50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- 172.21.15.57:0/1997140167 >> 172.21.15.57:0/1997140167 conn(0x55e392f53750 msgr2=0x55e392f60a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- 172.21.15.57:0/1997140167 shutdown_connections 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- 172.21.15.57:0/1997140167 wait complete. 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 -- start start 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.345+0000 7fe62ee3c740 1 Processor -- start 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f571e0 0x55e392f6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f6a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241af990 con 0x55e392f623a0 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241b0e00 con 0x55e392f57b00 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6241b0060 con 0x55e392f571e0 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62ee3c740 10 client.0 _ll_register_callbacks cb 0x7fe610005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.355 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46746/0 (socket says 172.21.15.57:46746) 2024-09-12T03:13:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe62a000640 1 -- 172.21.15.57:0/4047752704 learned_addr learned my addr 172.21.15.57:0/4047752704 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe628c00640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f6a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.346+0000 7fe629600640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f571e0 0x55e392f6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 -- 172.21.15.57:0/4047752704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f571e0 msgr2=0x55e392f6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.356 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f571e0 0x55e392f6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 -- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 msgr2=0x55e392f6a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f6a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 -- 172.21.15.57:0/4047752704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e392f4ed60 con 0x55e392f623a0 2024-09-12T03:13:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe628c00640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f6a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:13:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62a000640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009ef0 tx=0x7fe61c004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9424df6530fbe758 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe60aa00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004a20 con 0x55e392f623a0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7fe61c004bc0 con 0x55e392f623a0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61804f070 con 0x55e392f623a0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.347+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe61c004dc0 con 0x55e392f623a0 2024-09-12T03:13:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e392fdc420 con 0x55e392f623a0 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61804f070 con 0x55e392f623a0 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe61c00f480 con 0x55e392f623a0 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 20 client.15651 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392fdca00 0x55e392fdee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 4 client.15651 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:13:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e392ec4b80 con 0x55e392fdca00 2024-09-12T03:13:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.348+0000 7fe629600640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392fdca00 0x55e392fdee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.349+0000 7fe629600640 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392fdca00 0x55e392fdee20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9028b017fde18e5a server_cookie=5fcf2f5f3722f8fb in_seq=0 out_seq=0 2024-09-12T03:13:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.349+0000 7fe60aa00640 10 client.15651 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.353+0000 7fe60aa00640 1 -- 172.21.15.57:0/4047752704 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e392ec4b80 con 0x55e392fdca00 2024-09-12T03:13:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.353+0000 7fe60aa00640 10 client.15651 handle_command_reply: tid=0 2024-09-12T03:13:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.353+0000 7fe62ee3c740 1 client.15651 shutdown 2024-09-12T03:13:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 msgr2=0x55e392f5b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009ef0 tx=0x7fe61c004260 comp rx=0 tx=0).stop 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 shutdown_connections 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55e392fdca00 0x55e392fdee20 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55e392f623a0 0x55e392f5b910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55e392f57b00 0x55e392f6a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 --2- 172.21.15.57:0/4047752704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e392f571e0 0x55e392f6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 >> 172.21.15.57:0/4047752704 conn(0x55e392f53750 msgr2=0x55e392f5fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe60aa00640 0 client.15651 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 shutdown_connections 2024-09-12T03:13:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 1 -- 172.21.15.57:0/4047752704 wait complete. 2024-09-12T03:13:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.354+0000 7fe62ee3c740 20 client.15651 trim_cache size 0 max 16384 2024-09-12T03:13:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:13:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:13:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:13:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 msgr2=0x7fe60c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 0x7fe60c0a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe618005e00 tx=0x7fe6180056d0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 msgr2=0x7fe62419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 0x7fe62419aad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe610005fd0 tx=0x7fe61000ee50 comp rx=0 tx=0).stop 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 shutdown_connections 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe624107b80 0x7fe62419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe624102480 0x7fe62419aad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fe60c0a2da0 0x7fe60c0a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 --2- 172.21.15.57:0/845249839 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fe624101ae0 0x7fe62419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 >> 172.21.15.57:0/845249839 conn(0x7fe6240fd7e0 msgr2=0x7fe62410d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 shutdown_connections 2024-09-12T03:13:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:39.355+0000 7fe62c000640 1 -- 172.21.15.57:0/845249839 wait complete. 2024-09-12T03:13:40.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:39 smithi181 ceph-mon[35320]: pgmap v238: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:40.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:39 smithi057 ceph-mon[30361]: pgmap v238: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:40.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:39 smithi204 ceph-mon[35013]: pgmap v238: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-09-12T03:13:42.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:41 smithi181 ceph-mon[35320]: pgmap v239: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:13:42.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:41 smithi057 ceph-mon[30361]: pgmap v239: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:13:42.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:41 smithi204 ceph-mon[35013]: pgmap v239: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-12T03:13:44.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:43 smithi181 ceph-mon[35320]: pgmap v240: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T03:13:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:43 smithi057 ceph-mon[30361]: pgmap v240: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T03:13:44.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:43 smithi204 ceph-mon[35013]: pgmap v240: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-09-12T03:13:46.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:45 smithi181 ceph-mon[35320]: pgmap v241: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:45 smithi057 ceph-mon[30361]: pgmap v241: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:46.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:45 smithi204 ceph-mon[35013]: pgmap v241: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:47 smithi181 ceph-mon[35320]: pgmap v242: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:47 smithi057 ceph-mon[30361]: pgmap v242: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:48.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:47 smithi204 ceph-mon[35013]: pgmap v242: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:50.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:49 smithi181 ceph-mon[35320]: pgmap v243: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:50.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:49 smithi057 ceph-mon[30361]: pgmap v243: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:50.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:49 smithi204 ceph-mon[35013]: pgmap v243: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:51.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:50 smithi181 ceph-mon[35320]: pgmap v244: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:51.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:50 smithi204 ceph-mon[35013]: pgmap v244: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:51.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:50 smithi057 ceph-mon[30361]: pgmap v244: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.483+0000 7f3841000640 1 -- 172.21.15.57:0/2070959131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c109080 msgr2=0x7f383c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.483+0000 7f3841000640 1 --2- 172.21.15.57:0/2070959131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c109080 0x7f383c110bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3834009be0 tx=0x7f383402a6c0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 -- 172.21.15.57:0/2070959131 shutdown_connections 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 --2- 172.21.15.57:0/2070959131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c109080 0x7f383c110bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 --2- 172.21.15.57:0/2070959131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c1048e0 0x7f383c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 --2- 172.21.15.57:0/2070959131 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c103f30 0x7f383c104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 -- 172.21.15.57:0/2070959131 >> 172.21.15.57:0/2070959131 conn(0x7f383c0ff980 msgr2=0x7f383c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 -- 172.21.15.57:0/2070959131 shutdown_connections 2024-09-12T03:13:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.484+0000 7f3841000640 1 -- 172.21.15.57:0/2070959131 wait complete. 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 Processor -- start 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 -- start start 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 0x7f383c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 0x7f383c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 0x7f383c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f383c114090 con 0x7f383c109080 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f383c113f10 con 0x7f383c103f30 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3841000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c114210 con 0x7f383c19edc0 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3839400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 0x7f383c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.485+0000 7f3839e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 0x7f383c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f3839e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 0x7f383c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35956/0 (socket says 172.21.15.57:35956) 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 0x7f383c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f3839e00640 1 -- 172.21.15.57:0/2502215865 learned_addr learned my addr 172.21.15.57:0/2502215865 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f3839400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 0x7f383c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45978/0 (socket says 172.21.15.57:45978) 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 -- 172.21.15.57:0/2502215865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 msgr2=0x7f383c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 0x7f383c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 -- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 msgr2=0x7f383c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 0x7f383c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 -- 172.21.15.57:0/2502215865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383c19f8b0 con 0x7f383c19edc0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f383a800640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 0x7f383c19f270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3834034040 tx=0x7f38340305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.486+0000 7f3832a00640 1 -- 172.21.15.57:0/2502215865 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834041020 con 0x7f383c19edc0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.487+0000 7f3832a00640 1 -- 172.21.15.57:0/2502215865 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3834037040 con 0x7f383c19edc0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.487+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f383c19fb40 con 0x7f383c19edc0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.487+0000 7f3832a00640 1 -- 172.21.15.57:0/2502215865 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834036070 con 0x7f383c19edc0 2024-09-12T03:13:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.487+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f383c1a7300 con 0x7f383c19edc0 2024-09-12T03:13:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 Processor -- start 2024-09-12T03:13:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3832a00640 1 -- 172.21.15.57:0/2502215865 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f3834044020 con 0x7f383c19edc0 2024-09-12T03:13:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 -- start start 2024-09-12T03:13:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbe1e0 0x555fb6bbe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 0x555fb6bcb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555fb6b24100 con 0x555fb6bc93a0 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555fb6acc1b0 con 0x555fb6bbeb00 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.490+0000 7f3843ec1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fb6b200c0 con 0x555fb6bbe1e0 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 0x555fb6bcb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:35958/0 (socket says 172.21.15.57:35958) 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 -- 172.21.15.57:0/2006178799 learned_addr learned my addr 172.21.15.57:0/2006178799 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839400640 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbe1e0 0x555fb6bbe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 -- 172.21.15.57:0/2006178799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbe1e0 msgr2=0x555fb6bbe5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbe1e0 0x555fb6bbe5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 -- 172.21.15.57:0/2006178799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 msgr2=0x555fb6bcb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 0x555fb6bcb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 -- 172.21.15.57:0/2006178799 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb6837210 con 0x555fb6bbeb00 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839400640 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 0x555fb6bcb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:13:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3839e00640 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f382c009d80 tx=0x7f382c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678f5c774bc351de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.491+0000 7f3832a00640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 0x7f38200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.492+0000 7f3832a00640 1 -- 172.21.15.57:0/2502215865 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f38340f0220 con 0x7f383c19edc0 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.492+0000 7f3827400640 1 -- 172.21.15.57:0/2006178799 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c004230 con 0x555fb6bbeb00 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.492+0000 7f3827400640 1 -- 172.21.15.57:0/2006178799 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c0043d0 con 0x555fb6bbeb00 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.492+0000 7f3827400640 1 -- 172.21.15.57:0/2006178799 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c0198c0 con 0x555fb6bbeb00 2024-09-12T03:13:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.492+0000 7f3839400640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 0x7f38200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- 172.21.15.57:0/2006178799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 msgr2=0x555fb6bc8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3839400640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 0x7f38200a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x555fb6bbbad0 tx=0x7f38280096a0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f382c009d80 tx=0x7f382c011810 comp rx=0 tx=0).stop 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- 172.21.15.57:0/2006178799 shutdown_connections 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bc93a0 0x555fb6bcb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bbeb00 0x555fb6bc8e60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2006178799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbe1e0 0x555fb6bbe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- 172.21.15.57:0/2006178799 >> 172.21.15.57:0/2006178799 conn(0x555fb6bba750 msgr2=0x555fb6bbab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- 172.21.15.57:0/2006178799 shutdown_connections 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- 172.21.15.57:0/2006178799 wait complete. 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.493+0000 7f3843ec1740 1 -- start start 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 Processor -- start 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bc6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x555fb6bcbf70 con 0x555fb6bbe1e0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x555fb6aad090 con 0x555fb6bc93a0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fb6bcc0f0 con 0x555fb6bbeb00 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3843ec1740 10 client.0 _ll_register_callbacks cb 0x555fb6b01f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f383a800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bc6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3839400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46010/0 (socket says 172.21.15.57:46010) 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.494+0000 7f3839e00640 1 -- 172.21.15.57:0/570388560 learned_addr learned my addr 172.21.15.57:0/570388560 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 -- 172.21.15.57:0/570388560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 msgr2=0x555fb6bc6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bc6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 -- 172.21.15.57:0/570388560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 msgr2=0x555fb6bc62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 -- 172.21.15.57:0/570388560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb6903620 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f383a800640 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bc6d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38340ed4d0 tx=0x7f38340bbed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffeb9a0c09f2c5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f381be00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340c0680 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f38340b8940 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.495+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382c0043d0 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38340c4f40 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 10 client.25811 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555fb6bd7b70 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f382c0043d0 con 0x555fb6bc93a0 2024-09-12T03:13:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f38340c4ab0 con 0x555fb6bc93a0 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 10 client.25811 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 10 client.25811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 20 client.25811 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6bd9020 0x555fb6bdb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 4 client.25811 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.496+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fb6b31830 con 0x555fb6bd9020 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.497+0000 7f3839e00640 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6bd9020 0x555fb6bdb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.497+0000 7f3839e00640 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6bd9020 0x555fb6bdb450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8459de083e9ce88 server_cookie=a18c0ac0dbfe500a in_seq=0 out_seq=0 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.497+0000 7f381be00640 10 client.25811 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f381be00640 1 -- 172.21.15.57:0/570388560 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fb6b31830 con 0x555fb6bd9020 2024-09-12T03:13:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f381be00640 10 client.25811 handle_command_reply: tid=0 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 client.25811 shutdown 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 msgr2=0x555fb6bc6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bc6d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38340ed4d0 tx=0x7f38340bbed0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 shutdown_connections 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6bd9020 0x555fb6bdb450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bc6d60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bc6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f381be00640 0 client.25811 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 --2- 172.21.15.57:0/570388560 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 >> 172.21.15.57:0/570388560 conn(0x555fb6bba750 msgr2=0x555fb6bc0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 shutdown_connections 2024-09-12T03:13:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.498+0000 7f3843ec1740 1 -- 172.21.15.57:0/570388560 wait complete. 2024-09-12T03:13:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.499+0000 7f3843ec1740 20 client.25811 trim_cache size 0 max 16384 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 Processor -- start 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 -- start start 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6be0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6be12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b3b40 con 0x555fb6bbe1e0 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b1410 con 0x555fb6bc93a0 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3843ec1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1af280 con 0x555fb6bbeb00 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3839400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6be0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46022/0 (socket says 172.21.15.57:46022) 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.514+0000 7f3839e00640 1 -- 172.21.15.57:0/2456614774 learned_addr learned my addr 172.21.15.57:0/2456614774 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6be12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 -- 172.21.15.57:0/2456614774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 msgr2=0x555fb6be0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6be0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 -- 172.21.15.57:0/2456614774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 msgr2=0x555fb6bc8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 -- 172.21.15.57:0/2456614774 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb6bbe010 con 0x555fb6bc93a0 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f383a800640 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6be12a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38340c5100 tx=0x7f3834003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddb8f035ba648588 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f3826a00640 1 -- 172.21.15.57:0/2456614774 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834003b70 con 0x555fb6bc93a0 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f3826a00640 1 -- 172.21.15.57:0/2456614774 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38340c4ab0 con 0x555fb6bc93a0 2024-09-12T03:13:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.515+0000 7f3826a00640 1 -- 172.21.15.57:0/2456614774 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340ed780 con 0x555fb6bc93a0 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- 172.21.15.57:0/2456614774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 msgr2=0x555fb6be12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6be12a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f38340c5100 tx=0x7f3834003ea0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- 172.21.15.57:0/2456614774 shutdown_connections 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6be12a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6be0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 --2- 172.21.15.57:0/2456614774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bc8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- 172.21.15.57:0/2456614774 >> 172.21.15.57:0/2456614774 conn(0x555fb6bba750 msgr2=0x555fb6bc7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- 172.21.15.57:0/2456614774 shutdown_connections 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- 172.21.15.57:0/2456614774 wait complete. 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.516+0000 7f3843ec1740 1 -- start start 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 Processor -- start 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1af8d0 con 0x555fb6bbe1e0 2024-09-12T03:13:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1b0d40 con 0x555fb6bc93a0 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f383c1affa0 con 0x555fb6bbeb00 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3843ec1740 10 client.0 _ll_register_callbacks cb 0x7f38340bc0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3839e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:46034/0 (socket says 172.21.15.57:46034) 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.517+0000 7f3839e00640 1 -- 172.21.15.57:0/3329743776 learned_addr learned my addr 172.21.15.57:0/3329743776 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f383a800640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839400640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 -- 172.21.15.57:0/3329743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 msgr2=0x555fb6bd1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bd1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 -- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 msgr2=0x555fb6bce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 -- 172.21.15.57:0/3329743776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb6bb5d60 con 0x555fb6bbe1e0 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3839e00640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c8d38e9a50d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f381be00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c005650 con 0x555fb6bbe1e0 2024-09-12T03:13:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f382c01f040 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.518+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38340c4ab0 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f382c037050 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555fb6c44470 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38340c4ab0 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f382c0199d0 con 0x555fb6bbe1e0 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 20 client.15669 populate_metadata read hostname 'smithi057' 2024-09-12T03:13:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6c44a40 0x555fb6c46e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 4 client.15669 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:13:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.519+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fb6b2bb80 con 0x555fb6c44a40 2024-09-12T03:13:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.520+0000 7f3839400640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6c44a40 0x555fb6c46e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.520+0000 7f3839400640 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6c44a40 0x555fb6c46e30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18d81607c1f2ed5 server_cookie=e3797efb6d1ce105 in_seq=0 out_seq=0 2024-09-12T03:13:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.520+0000 7f381be00640 10 client.15669 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.524+0000 7f381be00640 1 -- 172.21.15.57:0/3329743776 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fb6b2bb80 con 0x555fb6c44a40 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.524+0000 7f381be00640 10 client.15669 handle_command_reply: tid=0 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.524+0000 7f3843ec1740 1 client.15669 shutdown 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 msgr2=0x555fb6bd1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f382c000c00 tx=0x7f382c009620 comp rx=0 tx=0).stop 2024-09-12T03:13:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 shutdown_connections 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x555fb6c44a40 0x555fb6c46e30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x555fb6bc93a0 0x555fb6bce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fb6bbeb00 0x555fb6bd1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 --2- 172.21.15.57:0/3329743776 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x555fb6bbe1e0 0x555fb6bd1340 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 >> 172.21.15.57:0/3329743776 conn(0x555fb6bba750 msgr2=0x555fb6bb9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f381be00640 0 client.15669 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 shutdown_connections 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 1 -- 172.21.15.57:0/3329743776 wait complete. 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.525+0000 7f3843ec1740 20 client.15669 trim_cache size 0 max 16384 2024-09-12T03:13:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:13:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:13:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:13:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:13:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 msgr2=0x7f38200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 0x7f38200a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x555fb6bbbad0 tx=0x7f38280096a0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 msgr2=0x7f383c19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 0x7f383c19f270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3834034040 tx=0x7f38340305d0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 shutdown_connections 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f383c19edc0 0x7f383c19f270 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f38200a2da0 0x7f38200a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f383c109080 0x7f383c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 --2- 172.21.15.57:0/2502215865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383c103f30 0x7f383c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 >> 172.21.15.57:0/2502215865 conn(0x7f383c0ff980 msgr2=0x7f383c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 shutdown_connections 2024-09-12T03:13:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:13:52.526+0000 7f3841000640 1 -- 172.21.15.57:0/2502215865 wait complete. 2024-09-12T03:13:53.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:53 smithi057 ceph-mon[30361]: pgmap v245: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:53.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:53 smithi204 ceph-mon[35013]: pgmap v245: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:53.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:53 smithi181 ceph-mon[35320]: pgmap v245: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:55.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:55 smithi057 ceph-mon[30361]: pgmap v246: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:13:55.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:55 smithi204 ceph-mon[35013]: pgmap v246: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:13:55.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:55 smithi181 ceph-mon[35320]: pgmap v246: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:13:57.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:57 smithi204 ceph-mon[35013]: pgmap v247: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:57.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:57 smithi181 ceph-mon[35320]: pgmap v247: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:57.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:57 smithi057 ceph-mon[30361]: pgmap v247: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:13:59.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:13:59 smithi204 ceph-mon[35013]: pgmap v248: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:59.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:13:59 smithi181 ceph-mon[35320]: pgmap v248: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:13:59.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:13:59 smithi057 ceph-mon[30361]: pgmap v248: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:01.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:01 smithi204 ceph-mon[35013]: pgmap v249: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:01.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:01 smithi181 ceph-mon[35320]: pgmap v249: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:01.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:01 smithi057 ceph-mon[30361]: pgmap v249: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:03.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:03 smithi204 ceph-mon[35013]: pgmap v250: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:14:03.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:03 smithi181 ceph-mon[35320]: pgmap v250: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:14:03.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:03 smithi057 ceph-mon[30361]: pgmap v250: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:14:05.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:05 smithi204 ceph-mon[35013]: pgmap v251: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.3 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:14:05.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:05 smithi181 ceph-mon[35320]: pgmap v251: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.3 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:14:05.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:05 smithi057 ceph-mon[30361]: pgmap v251: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 6.3 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:14:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 -- 172.21.15.57:0/1705764733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01018b0 msgr2=0x7f19f0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 --2- 172.21.15.57:0/1705764733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01018b0 0x7f19f0101d10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009930 tx=0x7f19dc02a420 comp rx=0 tx=0).stop 2024-09-12T03:14:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 -- 172.21.15.57:0/1705764733 shutdown_connections 2024-09-12T03:14:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 --2- 172.21.15.57:0/1705764733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f0102250 0x7f19f010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 --2- 172.21.15.57:0/1705764733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f01018b0 0x7f19f0101d10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 --2- 172.21.15.57:0/1705764733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01078c0 0x7f19f0107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 -- 172.21.15.57:0/1705764733 >> 172.21.15.57:0/1705764733 conn(0x7f19f00fd5d0 msgr2=0x7f19f00ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 -- 172.21.15.57:0/1705764733 shutdown_connections 2024-09-12T03:14:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.655+0000 7f19f4a00640 1 -- 172.21.15.57:0/1705764733 wait complete. 2024-09-12T03:14:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.656+0000 7f19f4a00640 1 Processor -- start 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 -- start start 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 0x7f19f019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 0x7f19f019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111d00 con 0x7f19f0102250 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111b80 con 0x7f19f01078c0 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19f4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111e80 con 0x7f19f01018b0 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19ef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 0x7f19f019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 0x7f19f019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19ef400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:47126/0 (socket says 172.21.15.57:47126) 2024-09-12T03:14:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.657+0000 7f19ef400640 1 -- 172.21.15.57:0/2024538505 learned_addr learned my addr 172.21.15.57:0/2024538505 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 -- 172.21.15.57:0/2024538505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 msgr2=0x7f19f019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 0x7f19f019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 -- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 msgr2=0x7f19f019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 0x7f19f019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 -- 172.21.15.57:0/2024538505 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f019f440 con 0x7f19f01018b0 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19efe00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 0x7f19f019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19eea00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 0x7f19f019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19ef400640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19e7e00640 1 -- 172.21.15.57:0/2024538505 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000cd80 con 0x7f19f01018b0 2024-09-12T03:14:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19e7e00640 1 -- 172.21.15.57:0/2024538505 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e0010910 con 0x7f19f01018b0 2024-09-12T03:14:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f019f730 con 0x7f19f01018b0 2024-09-12T03:14:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.658+0000 7f19e7e00640 1 -- 172.21.15.57:0/2024538505 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00188f0 con 0x7f19f01018b0 2024-09-12T03:14:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.659+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01a6fc0 con 0x7f19f01018b0 2024-09-12T03:14:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.661+0000 7f19f8ec8740 1 Processor -- start 2024-09-12T03:14:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 -- start start 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19e7e00640 1 -- 172.21.15.57:0/2024538505 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f19e0010ac0 con 0x7f19f01018b0 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf1e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf20960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558aeae79100 con 0x558aeaf133b0 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558aeae211b0 con 0x558aeaf13cd0 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558aeae750c0 con 0x558aeaf1e570 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19ef400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf1e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43506/0 (socket says 172.21.15.57:43506) 2024-09-12T03:14:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19efe00640 1 -- 172.21.15.57:0/3965510579 learned_addr learned my addr 172.21.15.57:0/3965510579 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.662+0000 7f19eea00640 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf20960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 -- 172.21.15.57:0/3965510579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 msgr2=0x558aeaf20960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf20960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 -- 172.21.15.57:0/3965510579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 msgr2=0x558aeaf1e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf1e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 -- 172.21.15.57:0/3965510579 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aeab8c210 con 0x558aeaf133b0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19efe00640 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f19e80117e0 tx=0x7f19e8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f93ca61814a2fca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19dbe00640 1 -- 172.21.15.57:0/3965510579 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8011310 con 0x558aeaf133b0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19dbe00640 1 -- 172.21.15.57:0/3965510579 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8003e50 con 0x558aeaf133b0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.663+0000 7f19dbe00640 1 -- 172.21.15.57:0/3965510579 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80045e0 con 0x558aeaf133b0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19e7e00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 0x7f19d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19e7e00640 1 -- 172.21.15.57:0/2024538505 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f19e00cfc20 con 0x7f19f01018b0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19ef400640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 0x7f19d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 -- 172.21.15.57:0/3965510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 msgr2=0x558aeaf13790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f19e80117e0 tx=0x7f19e8011ca0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 -- 172.21.15.57:0/3965510579 shutdown_connections 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf20960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf1e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/3965510579 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf133b0 0x558aeaf13790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 -- 172.21.15.57:0/3965510579 >> 172.21.15.57:0/3965510579 conn(0x558aeaf0f920 msgr2=0x558aeaf0fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.664+0000 7f19f8ec8740 1 -- 172.21.15.57:0/3965510579 shutdown_connections 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.665+0000 7f19f8ec8740 1 -- 172.21.15.57:0/3965510579 wait complete. 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.665+0000 7f19ef400640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 0x7f19d40a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x558aeaf10780 tx=0x7f19e00d36d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.665+0000 7f19f8ec8740 1 -- start start 2024-09-12T03:14:05.667 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.665+0000 7f19f8ec8740 1 Processor -- start 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf1b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf1c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x558aeaf20f90 con 0x558aeaf13cd0 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x558aeae02090 con 0x558aeaf133b0 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558aeaf21110 con 0x558aeaf1e570 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19f8ec8740 10 client.0 _ll_register_callbacks cb 0x558aeae56f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:43514/0 (socket says 172.21.15.57:43514) 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 -- 172.21.15.57:0/2961439941 learned_addr learned my addr 172.21.15.57:0/2961439941 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19efe00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf1b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19efe00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf1c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 -- 172.21.15.57:0/2961439941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 msgr2=0x558aeaf1c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf1c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 -- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 msgr2=0x558aeaf1b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf1b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19eea00640 1 -- 172.21.15.57:0/2961439941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aeac58620 con 0x558aeaf13cd0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.666+0000 7f19efe00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf1b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19eea00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19dc02a940 tx=0x7f19dc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f860ca01535cf9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc035040 con 0x558aeaf13cd0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e8003e50 con 0x558aeaf13cd0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f19dc004450 con 0x558aeaf13cd0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19dc0035f0 con 0x558aeaf13cd0 2024-09-12T03:14:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19f8ec8740 10 client.15681 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.667+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558aeaf2cde0 con 0x558aeaf13cd0 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8003e50 con 0x558aeaf13cd0 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f19dc048950 con 0x558aeaf13cd0 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 10 client.15681 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 20 client.15681 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf2e250 0x558aeaf30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 4 client.15681 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aeae86830 con 0x558aeaf2e250 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.668+0000 7f19efe00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf2e250 0x558aeaf30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.669+0000 7f19efe00640 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf2e250 0x558aeaf30640 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e23a81e3557fdf3 server_cookie=47c53fd92eb1cff1 in_seq=0 out_seq=0 2024-09-12T03:14:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.669+0000 7f19d8c00640 10 client.15681 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19d8c00640 1 -- 172.21.15.57:0/2961439941 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558aeae86830 con 0x558aeaf2e250 2024-09-12T03:14:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19d8c00640 10 client.15681 handle_command_reply: tid=0 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 client.15681 shutdown 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 msgr2=0x558aeaf1bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19dc02a940 tx=0x7f19dc0317e0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 shutdown_connections 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf2e250 0x558aeaf30640 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf1e570 0x558aeaf1c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf13cd0 0x558aeaf1bb80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19d8c00640 0 client.15681 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/2961439941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf1b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 >> 172.21.15.57:0/2961439941 conn(0x558aeaf0f920 msgr2=0x558aeaf15680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 shutdown_connections 2024-09-12T03:14:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 1 -- 172.21.15.57:0/2961439941 wait complete. 2024-09-12T03:14:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.670+0000 7f19f8ec8740 20 client.15681 trim_cache size 0 max 16384 2024-09-12T03:14:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 Processor -- start 2024-09-12T03:14:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 -- start start 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf13cd0 0x558aeaf1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf35f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01b3b50 con 0x558aeaf1e570 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01b1420 con 0x558aeaf133b0 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01af290 con 0x558aeaf13cd0 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:57896/0 (socket says 172.21.15.57:57896) 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.686+0000 7f19efe00640 1 -- 172.21.15.57:0/1006926415 learned_addr learned my addr 172.21.15.57:0/1006926415 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf35f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19eea00640 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf13cd0 0x558aeaf1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 -- 172.21.15.57:0/1006926415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf13cd0 msgr2=0x558aeaf1d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf13cd0 0x558aeaf1d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 -- 172.21.15.57:0/1006926415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 msgr2=0x558aeaf35f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf35f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 -- 172.21.15.57:0/1006926415 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aeaf105b0 con 0x558aeaf133b0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19efe00640 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19e8004270 tx=0x7f19e801d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5355675cca96575 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.687+0000 7f19db400640 1 -- 172.21.15.57:0/1006926415 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8025040 con 0x558aeaf133b0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19db400640 1 -- 172.21.15.57:0/1006926415 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8021070 con 0x558aeaf133b0 2024-09-12T03:14:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19db400640 1 -- 172.21.15.57:0/1006926415 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8005790 con 0x558aeaf133b0 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 -- 172.21.15.57:0/1006926415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 msgr2=0x558aeaf15e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f19e8004270 tx=0x7f19e801d040 comp rx=0 tx=0).stop 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 -- 172.21.15.57:0/1006926415 shutdown_connections 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf35f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf13cd0 0x558aeaf1d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/1006926415 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf133b0 0x558aeaf15e20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 -- 172.21.15.57:0/1006926415 >> 172.21.15.57:0/1006926415 conn(0x558aeaf0f920 msgr2=0x558aeaf1cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 -- 172.21.15.57:0/1006926415 shutdown_connections 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.688+0000 7f19f8ec8740 1 -- 172.21.15.57:0/1006926415 wait complete. 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 -- start start 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 Processor -- start 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf133b0 0x558aeaf18560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01af8e0 con 0x558aeaf1e570 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01b0d50 con 0x558aeaf13cd0 2024-09-12T03:14:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01affb0 con 0x558aeaf133b0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.689+0000 7f19f8ec8740 10 client.0 _ll_register_callbacks cb 0x7f19dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:57904/0 (socket says 172.21.15.57:57904) 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 -- 172.21.15.57:0/928393346 learned_addr learned my addr 172.21.15.57:0/928393346 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19efe00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19efe00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf133b0 0x558aeaf18560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 -- 172.21.15.57:0/928393346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf133b0 msgr2=0x558aeaf18560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf133b0 0x558aeaf18560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 -- 172.21.15.57:0/928393346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 msgr2=0x558aeaf18fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf18fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 -- 172.21.15.57:0/928393346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aeaf97ee0 con 0x558aeaf13cd0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19eea00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000c00 tx=0x7f19dc031cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4447023992ab5dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.690+0000 7f19d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc035070 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e8021070 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9554+0+0 (secure 0 0 0) 0x7f19dc048890 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19dc005d50 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 10 client.25835 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558aeaf99160 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8021070 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f19dc05c070 con 0x558aeaf13cd0 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 10 client.25835 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 10 client.25835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 20 client.25835 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.691+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf99930 0x558aeaf9bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.692+0000 7f19f8ec8740 4 client.25835 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:14:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.692+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558aeae80b80 con 0x558aeaf99930 2024-09-12T03:14:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.692+0000 7f19efe00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf99930 0x558aeaf9bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.692+0000 7f19efe00640 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf99930 0x558aeaf9bd50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed33fa79aa365911 server_cookie=98e2316ef0b07f73 in_seq=0 out_seq=0 2024-09-12T03:14:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.692+0000 7f19d8c00640 10 client.25835 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19d8c00640 1 -- 172.21.15.57:0/928393346 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558aeae80b80 con 0x558aeaf99930 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19d8c00640 10 client.25835 handle_command_reply: tid=0 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 client.25835 shutdown 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 msgr2=0x558aeaf18aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f19dc000c00 tx=0x7f19dc031cc0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 shutdown_connections 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x558aeaf99930 0x558aeaf9bd50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x558aeaf1e570 0x558aeaf18fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558aeaf13cd0 0x558aeaf18aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 --2- 172.21.15.57:0/928393346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558aeaf133b0 0x558aeaf18560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19d8c00640 0 client.25835 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 >> 172.21.15.57:0/928393346 conn(0x558aeaf0f920 msgr2=0x558aeaf29a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.699+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 shutdown_connections 2024-09-12T03:14:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f8ec8740 1 -- 172.21.15.57:0/928393346 wait complete. 2024-09-12T03:14:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f8ec8740 20 client.25835 trim_cache size 0 max 16384 2024-09-12T03:14:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:14:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:14:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:14:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 msgr2=0x7f19d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 0x7f19d40a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x558aeaf10780 tx=0x7f19e00d36d0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 msgr2=0x7f19f019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 shutdown_connections 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19f01078c0 0x7f19f019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f19d40a2cf0 0x7f19d40a51b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f19f0102250 0x7f19f019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 --2- 172.21.15.57:0/2024538505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19f01018b0 0x7f19f019a450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 >> 172.21.15.57:0/2024538505 conn(0x7f19f00fd5d0 msgr2=0x7f19f010ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.700+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 shutdown_connections 2024-09-12T03:14:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:05.701+0000 7f19f4a00640 1 -- 172.21.15.57:0/2024538505 wait complete. 2024-09-12T03:14:07.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:07 smithi181 ceph-mon[35320]: pgmap v252: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:14:07.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:07 smithi057 ceph-mon[30361]: pgmap v252: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:14:07.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:07 smithi204 ceph-mon[35013]: pgmap v252: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:14:08.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:08 smithi181 ceph-mon[35320]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:14:08.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:08 smithi057 ceph-mon[30361]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:14:08.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:08 smithi204 ceph-mon[35013]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T03:14:09.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:09 smithi181 ceph-mon[35320]: pgmap v254: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:09.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:09 smithi057 ceph-mon[30361]: pgmap v254: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:09 smithi204 ceph-mon[35013]: pgmap v254: 129 pgs: 129 active+clean; 44 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:11.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:11 smithi181 ceph-mon[35320]: pgmap v255: 129 pgs: 1 active+clean+snaptrim_wait, 128 active+clean; 43 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:11 smithi057 ceph-mon[30361]: pgmap v255: 129 pgs: 1 active+clean+snaptrim_wait, 128 active+clean; 43 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:11 smithi204 ceph-mon[35013]: pgmap v255: 129 pgs: 1 active+clean+snaptrim_wait, 128 active+clean; 43 GiB data, 91 GiB used, 981 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:14:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:13 smithi181 ceph-mon[35320]: pgmap v256: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:13 smithi057 ceph-mon[30361]: pgmap v256: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:13 smithi204 ceph-mon[35013]: pgmap v256: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:15.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:15 smithi181 ceph-mon[35320]: pgmap v257: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:15.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:15 smithi057 ceph-mon[30361]: pgmap v257: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:15 smithi204 ceph-mon[35013]: pgmap v257: 129 pgs: 1 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 127 active+clean; 36 GiB data, 78 GiB used, 995 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:14:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:17 smithi057 ceph-mon[30361]: pgmap v258: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-12T03:14:17.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:17 smithi057 ceph-mon[30361]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:14:17.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:17 smithi204 ceph-mon[35013]: pgmap v258: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-12T03:14:17.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:17 smithi204 ceph-mon[35013]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:14:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:17 smithi181 ceph-mon[35320]: pgmap v258: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2024-09-12T03:14:17.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:17 smithi181 ceph-mon[35320]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T03:14:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.818+0000 7f8c11a00640 1 -- 172.21.15.57:0/4152873782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c1048e0 msgr2=0x7f8c0c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.818+0000 7f8c11a00640 1 --2- 172.21.15.57:0/4152873782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c1048e0 0x7f8c0c108920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009860 tx=0x7f8bf802a340 comp rx=0 tx=0).stop 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 -- 172.21.15.57:0/4152873782 shutdown_connections 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 --2- 172.21.15.57:0/4152873782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c109080 0x7f8c0c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 --2- 172.21.15.57:0/4152873782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c1048e0 0x7f8c0c108920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 --2- 172.21.15.57:0/4152873782 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c103f30 0x7f8c0c104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 -- 172.21.15.57:0/4152873782 >> 172.21.15.57:0/4152873782 conn(0x7f8c0c0ff980 msgr2=0x7f8c0c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 -- 172.21.15.57:0/4152873782 shutdown_connections 2024-09-12T03:14:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.819+0000 7f8c11a00640 1 -- 172.21.15.57:0/4152873782 wait complete. 2024-09-12T03:14:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 Processor -- start 2024-09-12T03:14:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 -- start start 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c103f30 0x7f8c0c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c19ee50 0x7f8c0c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c114080 con 0x7f8c0c109080 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c113f00 con 0x7f8c0c103f30 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c11a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c114200 con 0x7f8c0c19ee50 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c0a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c0a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53680/0 (socket says 172.21.15.57:53680) 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c0a000640 1 -- 172.21.15.57:0/82836182 learned_addr learned my addr 172.21.15.57:0/82836182 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c103f30 0x7f8c0c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.820+0000 7f8c0b400640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c19ee50 0x7f8c0c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 -- 172.21.15.57:0/82836182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c19ee50 msgr2=0x7f8c0c19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c19ee50 0x7f8c0c19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 -- 172.21.15.57:0/82836182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c103f30 msgr2=0x7f8c0c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c103f30 0x7f8c0c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 -- 172.21.15.57:0/82836182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c19f940 con 0x7f8c0c109080 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c0a000640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8035040 tx=0x7f8bf8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c03400640 1 -- 172.21.15.57:0/82836182 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8037040 con 0x7f8c0c109080 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c19fbd0 con 0x7f8c0c109080 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.821+0000 7f8c03400640 1 -- 172.21.15.57:0/82836182 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8040070 con 0x7f8c0c109080 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.822+0000 7f8c03400640 1 -- 172.21.15.57:0/82836182 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8003a90 con 0x7f8c0c109080 2024-09-12T03:14:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.822+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1a73a0 con 0x7f8c0c109080 2024-09-12T03:14:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.825+0000 7f8c14a59740 1 Processor -- start 2024-09-12T03:14:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 -- start start 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c03400640 1 -- 172.21.15.57:0/82836182 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f8bf8003c30 con 0x7f8c0c109080 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5558dae9a100 con 0x5558daf3f3a0 2024-09-12T03:14:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5558dae421b0 con 0x5558daf341e0 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.826+0000 7f8c14a59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5558dae960c0 con 0x5558daf34b00 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48986/0 (socket says 172.21.15.57:48986) 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 -- 172.21.15.57:0/261301906 learned_addr learned my addr 172.21.15.57:0/261301906 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38116/0 (socket says 172.21.15.57:38116) 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 -- 172.21.15.57:0/261301906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 msgr2=0x5558daf3ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 -- 172.21.15.57:0/261301906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 msgr2=0x5558daf41790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf41790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 -- 172.21.15.57:0/261301906 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558dabad210 con 0x5558daf341e0 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8c0b400640 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009f70 tx=0x7f8c04010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4629fe49c5446c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.827+0000 7f8bf7e00640 1 -- 172.21.15.57:0/261301906 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04016070 con 0x5558daf341e0 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8bf7e00640 1 -- 172.21.15.57:0/261301906 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c04004d20 con 0x5558daf341e0 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8bf7e00640 1 -- 172.21.15.57:0/261301906 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04012950 con 0x5558daf341e0 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c03400640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 0x7f8bf00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c03400640 1 -- 172.21.15.57:0/82836182 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f8bf80ef800 con 0x7f8c0c109080 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 0x7f8bf00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 -- 172.21.15.57:0/261301906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 msgr2=0x5558daf345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009f70 tx=0x7f8c04010800 comp rx=0 tx=0).stop 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 -- 172.21.15.57:0/261301906 shutdown_connections 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf41790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf3ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.828+0000 7f8c14a59740 1 --2- 172.21.15.57:0/261301906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf345c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c14a59740 1 -- 172.21.15.57:0/261301906 >> 172.21.15.57:0/261301906 conn(0x5558daf30750 msgr2=0x5558daf30b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c14a59740 1 -- 172.21.15.57:0/261301906 shutdown_connections 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c14a59740 1 -- 172.21.15.57:0/261301906 wait complete. 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 0x7f8bf00a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5558daf315b0 tx=0x7f8bfc009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c14a59740 1 -- start start 2024-09-12T03:14:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.829+0000 7f8c14a59740 1 Processor -- start 2024-09-12T03:14:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf34b00 0x5558daf3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf3f3a0 0x5558daf3cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5558daf41f70 con 0x5558daf34b00 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5558dae23090 con 0x5558daf341e0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5558daf420f0 con 0x5558daf3f3a0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c14a59740 10 client.0 _ll_register_callbacks cb 0x5558dae77f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48994/0 (socket says 172.21.15.57:48994) 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c0aa00640 1 -- 172.21.15.57:0/2828779417 learned_addr learned my addr 172.21.15.57:0/2828779417 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c0a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf34b00 0x5558daf3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.830+0000 7f8c0b400640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf3f3a0 0x5558daf3cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 -- 172.21.15.57:0/2828779417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf3f3a0 msgr2=0x5558daf3cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf3f3a0 0x5558daf3cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 -- 172.21.15.57:0/2828779417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf34b00 msgr2=0x5558daf3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf34b00 0x5558daf3c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 -- 172.21.15.57:0/2828779417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558dac79620 con 0x5558daf341e0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc01ea00 tx=0x7f8bfc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2ec4630a042d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8bf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc004020 con 0x5558daf341e0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f8bfc018030 con 0x5558daf341e0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.831+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c04004d20 con 0x5558daf341e0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc034050 con 0x5558daf341e0 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 10 client.25847 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558daf4dc50 con 0x5558daf341e0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c04004d20 con 0x5558daf341e0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8bfc029420 con 0x5558daf341e0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 10 client.25847 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 10 client.25847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 20 client.25847 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558daf4f110 0x5558daf51560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 4 client.25847 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.832+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558daea7830 con 0x5558daf4f110 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.833+0000 7f8c0a000640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558daf4f110 0x5558daf51560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.833+0000 7f8c0a000640 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558daf4f110 0x5558daf51560 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabfbad8980da61 server_cookie=869401aba3a593a9 in_seq=0 out_seq=0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.833+0000 7f8bf4c00640 10 client.25847 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8bf4c00640 1 -- 172.21.15.57:0/2828779417 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558daea7830 con 0x5558daf4f110 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8bf4c00640 10 client.25847 handle_command_reply: tid=0 2024-09-12T03:14:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 client.25847 shutdown 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 msgr2=0x5558daf3c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc01ea00 tx=0x7f8bfc004730 comp rx=0 tx=0).stop 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 shutdown_connections 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558daf4f110 0x5558daf51560 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf3f3a0 0x5558daf3cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf34b00 0x5558daf3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8bf4c00640 0 client.25847 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 --2- 172.21.15.57:0/2828779417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf3c370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 >> 172.21.15.57:0/2828779417 conn(0x5558daf30750 msgr2=0x5558daf36460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 shutdown_connections 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 1 -- 172.21.15.57:0/2828779417 wait complete. 2024-09-12T03:14:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.834+0000 7f8c14a59740 20 client.25847 trim_cache size 0 max 16384 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 Processor -- start 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 -- start start 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf36c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1b3bf0 con 0x5558daf3f3a0 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1b14c0 con 0x5558daf341e0 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.850+0000 7f8c14a59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1af330 con 0x5558daf34b00 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0b400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf36c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:49000/0 (socket says 172.21.15.57:49000) 2024-09-12T03:14:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 -- 172.21.15.57:0/3294556212 learned_addr learned my addr 172.21.15.57:0/3294556212 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 -- 172.21.15.57:0/3294556212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 msgr2=0x5558daf36c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf36c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 -- 172.21.15.57:0/3294556212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 msgr2=0x5558daf56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 -- 172.21.15.57:0/3294556212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558daf3b300 con 0x5558daf341e0 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0b400640 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.851+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009530 tx=0x7f8bfc0295b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2713fc6e05f3ed2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.852+0000 7f8bf7400640 1 -- 172.21.15.57:0/3294556212 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc03c070 con 0x5558daf341e0 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.852+0000 7f8bf7400640 1 -- 172.21.15.57:0/3294556212 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc029e50 con 0x5558daf341e0 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.852+0000 7f8bf7400640 1 -- 172.21.15.57:0/3294556212 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc003970 con 0x5558daf341e0 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.852+0000 7f8c14a59740 1 -- 172.21.15.57:0/3294556212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 msgr2=0x5558daf2b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.852+0000 7f8c14a59740 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009530 tx=0x7f8bfc0295b0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 -- 172.21.15.57:0/3294556212 shutdown_connections 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf3f3a0 0x5558daf56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf36c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 --2- 172.21.15.57:0/3294556212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf341e0 0x5558daf2b8d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 -- 172.21.15.57:0/3294556212 >> 172.21.15.57:0/3294556212 conn(0x5558daf30750 msgr2=0x5558daf3e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 -- 172.21.15.57:0/3294556212 shutdown_connections 2024-09-12T03:14:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 -- 172.21.15.57:0/3294556212 wait complete. 2024-09-12T03:14:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 -- start start 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.853+0000 7f8c14a59740 1 Processor -- start 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf43e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf3f3a0 0x5558dafb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1af980 con 0x5558daf341e0 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1b0df0 con 0x5558daf3f3a0 2024-09-12T03:14:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1b0050 con 0x5558daf34b00 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c14a59740 10 client.0 _ll_register_callbacks cb 0x7f8bfc01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c0aa00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:53716/0 (socket says 172.21.15.57:53716) 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c0aa00640 1 -- 172.21.15.57:0/1698857635 learned_addr learned my addr 172.21.15.57:0/1698857635 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c0b400640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf3f3a0 0x5558dafb2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.854+0000 7f8c0a000640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf43e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 -- 172.21.15.57:0/1698857635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 msgr2=0x5558daf43e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf43e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 -- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf3f3a0 msgr2=0x5558dafb2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf3f3a0 0x5558dafb2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 -- 172.21.15.57:0/1698857635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558dafb34a0 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c0aa00640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc029d00 tx=0x7f8bfc0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bc33aa22d79518 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8bf4c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc03c040 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f8bfc005250 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.855+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bfc029e50 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bfc052050 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5558dafba800 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc029e50 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8bfc004c10 con 0x5558daf341e0 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 20 client.15705 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558dafbae00 0x5558dafbd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 4 client.15705 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.856+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558daea1b80 con 0x5558dafbae00 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.857+0000 7f8c0a000640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558dafbae00 0x5558dafbd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.857+0000 7f8c0a000640 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558dafbae00 0x5558dafbd1f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608359e39cf91181 server_cookie=be601feab8214b66 in_seq=0 out_seq=0 2024-09-12T03:14:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.857+0000 7f8bf4c00640 10 client.15705 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8bf4c00640 1 -- 172.21.15.57:0/1698857635 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558daea1b80 con 0x5558dafbae00 2024-09-12T03:14:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8bf4c00640 10 client.15705 handle_command_reply: tid=0 2024-09-12T03:14:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 client.15705 shutdown 2024-09-12T03:14:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 msgr2=0x5558daf43930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc029d00 tx=0x7f8bfc0042d0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 shutdown_connections 2024-09-12T03:14:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5558dafbae00 0x5558dafbd1f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5558daf3f3a0 0x5558dafb2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8bf4c00640 0 client.15705 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5558daf34b00 0x5558daf43e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 --2- 172.21.15.57:0/1698857635 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5558daf341e0 0x5558daf43930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 >> 172.21.15.57:0/1698857635 conn(0x5558daf30750 msgr2=0x5558daf3d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 shutdown_connections 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 1 -- 172.21.15.57:0/1698857635 wait complete. 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.863+0000 7f8c14a59740 20 client.15705 trim_cache size 0 max 16384 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:14:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 msgr2=0x7f8bf00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 0x7f8bf00a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5558daf315b0 tx=0x7f8bfc009260 comp rx=0 tx=0).stop 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 msgr2=0x7f8c0c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8035040 tx=0x7f8bf8033d90 comp rx=0 tx=0).stop 2024-09-12T03:14:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 shutdown_connections 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c0c19ee50 0x7f8c0c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8bf00a2da0 0x7f8bf00a5260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8c0c109080 0x7f8c0c19aad0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 --2- 172.21.15.57:0/82836182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c0c103f30 0x7f8c0c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 >> 172.21.15.57:0/82836182 conn(0x7f8c0c0ff980 msgr2=0x7f8c0c100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 shutdown_connections 2024-09-12T03:14:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:18.864+0000 7f8c11a00640 1 -- 172.21.15.57:0/82836182 wait complete. 2024-09-12T03:14:18.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:18 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:18.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:18 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:18.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:18 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:19 smithi057 ceph-mon[30361]: pgmap v260: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:19 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:19 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:19.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:19 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:14:19.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:19 smithi204 ceph-mon[35013]: pgmap v260: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:19.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:19 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:19.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:19 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:19.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:19 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:14:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:19 smithi181 ceph-mon[35320]: pgmap v260: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:19 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:19 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:19.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:19 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:14:21.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:21 smithi204 ceph-mon[35013]: pgmap v261: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:21 smithi181 ceph-mon[35320]: pgmap v261: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:21 smithi057 ceph-mon[30361]: pgmap v261: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:14:23.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:23 smithi204 ceph-mon[35013]: pgmap v262: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:23.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:23 smithi181 ceph-mon[35320]: pgmap v262: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:23 smithi057 ceph-mon[30361]: pgmap v262: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:25.922 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:25 smithi057 ceph-mon[30361]: pgmap v263: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:25.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:25 smithi181 ceph-mon[35320]: pgmap v263: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:26.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:25 smithi204 ceph-mon[35013]: pgmap v263: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:27.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:27 smithi181 ceph-mon[35320]: pgmap v264: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:28.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:27 smithi057 ceph-mon[30361]: pgmap v264: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:28.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:27 smithi204 ceph-mon[35013]: pgmap v264: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:14:29.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:29 smithi181 ceph-mon[35320]: pgmap v265: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:14:30.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:29 smithi057 ceph-mon[30361]: pgmap v265: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:14:30.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:29 smithi204 ceph-mon[35013]: pgmap v265: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:14:31.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:31 smithi181 ceph-mon[35320]: pgmap v266: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 -- 172.21.15.57:0/183429721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9210102250 msgr2=0x7f921010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 --2- 172.21.15.57:0/183429721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9210102250 0x7f921010e780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9208009cd0 tx=0x7f920802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 -- 172.21.15.57:0/183429721 shutdown_connections 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 --2- 172.21.15.57:0/183429721 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f9210102250 0x7f921010e780 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 --2- 172.21.15.57:0/183429721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f9210101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 --2- 172.21.15.57:0/183429721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92101078c0 0x7f9210107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.992+0000 7f9214a00640 1 -- 172.21.15.57:0/183429721 >> 172.21.15.57:0/183429721 conn(0x7f92100fd5d0 msgr2=0x7f92100ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.993+0000 7f9214a00640 1 -- 172.21.15.57:0/183429721 shutdown_connections 2024-09-12T03:14:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.993+0000 7f9214a00640 1 -- 172.21.15.57:0/183429721 wait complete. 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.993+0000 7f9214a00640 1 Processor -- start 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 -- start start 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f921019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 0x7f921019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111d50 con 0x7f92101078c0 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111bd0 con 0x7f92101018b0 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f9214a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9210111ed0 con 0x7f9210102250 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f921019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 0x7f921019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 0x7f921019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39358/0 (socket says 172.21.15.57:39358) 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f921019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50640/0 (socket says 172.21.15.57:50640) 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:39990/0 (socket says 172.21.15.57:39990) 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.994+0000 7f920fe00640 1 -- 172.21.15.57:0/4134205408 learned_addr learned my addr 172.21.15.57:0/4134205408 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 -- 172.21.15.57:0/4134205408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 msgr2=0x7f921019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f921019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 -- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 msgr2=0x7f921019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 0x7f921019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 -- 172.21.15.57:0/4134205408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921019f430 con 0x7f9210102250 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f920ea00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00e9f0 tx=0x7f91fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.995+0000 7f9207e00640 1 -- 172.21.15.57:0/4134205408 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00cd80 con 0x7f9210102250 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.996+0000 7f9207e00640 1 -- 172.21.15.57:0/4134205408 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc0035f0 con 0x7f9210102250 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.996+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921019f720 con 0x7f9210102250 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.996+0000 7f9207e00640 1 -- 172.21.15.57:0/4134205408 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc010690 con 0x7f9210102250 2024-09-12T03:14:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.996+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101a6fb0 con 0x7f9210102250 2024-09-12T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 Processor -- start 2024-09-12T03:14:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 -- start start 2024-09-12T03:14:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a89a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a89a570 0x55705a89c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55705a7f5100 con 0x55705a88fcd0 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55705a79d1b0 con 0x55705a89a570 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9218e9d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55705a7f10c0 con 0x55705a88f3b0 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f920fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f920fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:40000/0 (socket says 172.21.15.57:40000) 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f920fe00640 1 -- 172.21.15.57:0/2060132561 learned_addr learned my addr 172.21.15.57:0/2060132561 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f920f400640 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a89a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:31.999+0000 7f9207e00640 1 -- 172.21.15.57:0/4134205408 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f91fc003b30 con 0x7f9210102250 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 -- 172.21.15.57:0/2060132561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a89a570 msgr2=0x55705a89c960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:14:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a89a570 0x55705a89c960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 -- 172.21.15.57:0/2060132561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 msgr2=0x55705a89a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a89a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 -- 172.21.15.57:0/2060132561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705a508210 con 0x55705a88f3b0 2024-09-12T03:14:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f920fe00640 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9208009a50 tx=0x7f9208005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ace45006cf4809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.000+0000 7f91fbe00640 1 -- 172.21.15.57:0/2060132561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208049070 con 0x55705a88f3b0 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f91fbe00640 1 -- 172.21.15.57:0/2060132561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208036040 con 0x55705a88f3b0 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f91fbe00640 1 -- 172.21.15.57:0/2060132561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208003e60 con 0x55705a88f3b0 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9207e00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 0x7f91f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 -- 172.21.15.57:0/2060132561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 msgr2=0x55705a88f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9208009a50 tx=0x7f9208005ef0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9207e00640 1 -- 172.21.15.57:0/4134205408 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f91fc0cfab0 con 0x7f9210102250 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f920f400640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 0x7f91f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 -- 172.21.15.57:0/2060132561 shutdown_connections 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a89a570 0x55705a89c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a89a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 --2- 172.21.15.57:0/2060132561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a88f790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 -- 172.21.15.57:0/2060132561 >> 172.21.15.57:0/2060132561 conn(0x55705a88b920 msgr2=0x55705a88bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.001+0000 7f9218e9d740 1 -- 172.21.15.57:0/2060132561 shutdown_connections 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.002+0000 7f9218e9d740 1 -- 172.21.15.57:0/2060132561 wait complete. 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.002+0000 7f920f400640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 0x7f91f40a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55705a88c780 tx=0x7f9200005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.002+0000 7f9218e9d740 1 -- start start 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.002+0000 7f9218e9d740 1 Processor -- start 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 0x55705a897690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a89a570 0x55705a898110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55705a89cf90 con 0x55705a88fcd0 2024-09-12T03:14:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55705a77e090 con 0x55705a88f3b0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55705a89d110 con 0x55705a89a570 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f9218e9d740 10 client.0 _ll_register_callbacks cb 0x55705a7d2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 0x55705a897690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a89a570 0x55705a898110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39382/0 (socket says 172.21.15.57:39382) 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 -- 172.21.15.57:0/16478299 learned_addr learned my addr 172.21.15.57:0/16478299 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920f400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 0x55705a897690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50662/0 (socket says 172.21.15.57:50662) 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 -- 172.21.15.57:0/16478299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a89a570 msgr2=0x55705a898110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a89a570 0x55705a898110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 -- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 msgr2=0x55705a897690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 0x55705a897690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.003+0000 7f920ea00640 1 -- 172.21.15.57:0/16478299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705a5d4620 con 0x55705a88fcd0 2024-09-12T03:14:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f920ea00640 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f91fc01f000 tx=0x7f91fc09b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3fab3ee90c203c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f91f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc09bbc0 con 0x55705a88fcd0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9208036040 con 0x55705a88fcd0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f91fc098230 con 0x55705a88fcd0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91fc09ff70 con 0x55705a88fcd0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f9218e9d740 10 client.15717 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.004+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55705a8a8ed0 con 0x55705a88fcd0 2024-09-12T03:14:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208036040 con 0x55705a88fcd0 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f91fc09aa80 con 0x55705a88fcd0 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 10 client.15717 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 20 client.15717 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a8aa340 0x55705a8ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 4 client.15717 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55705a802830 con 0x55705a8aa340 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.005+0000 7f920f400640 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a8aa340 0x55705a8ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.006+0000 7f920f400640 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a8aa340 0x55705a8ac730 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea411b2307d914f server_cookie=61d0130dbfe716fa in_seq=0 out_seq=0 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.006+0000 7f91f8c00640 10 client.15717 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f91f8c00640 1 -- 172.21.15.57:0/16478299 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55705a802830 con 0x55705a8aa340 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f91f8c00640 10 client.15717 handle_command_reply: tid=0 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 client.15717 shutdown 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 msgr2=0x55705a897bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f91fc01f000 tx=0x7f91fc09b5a0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 shutdown_connections 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a8aa340 0x55705a8ac730 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a89a570 0x55705a898110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a88fcd0 0x55705a897bd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f91f8c00640 0 client.15717 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 --2- 172.21.15.57:0/16478299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88f3b0 0x55705a897690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 >> 172.21.15.57:0/16478299 conn(0x55705a88b920 msgr2=0x55705a891680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 shutdown_connections 2024-09-12T03:14:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.007+0000 7f9218e9d740 1 -- 172.21.15.57:0/16478299 wait complete. 2024-09-12T03:14:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.008+0000 7f9218e9d740 20 client.15717 trim_cache size 0 max 16384 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 Processor -- start 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 -- start start 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a891e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a8b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92101b3920 con 0x55705a89a570 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92101b11f0 con 0x55705a88fcd0 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f9218e9d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92101af060 con 0x55705a88f3b0 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f920ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f920fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a8b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f920ea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:50666/0 (socket says 172.21.15.57:50666) 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f920ea00640 1 -- 172.21.15.57:0/3965643350 learned_addr learned my addr 172.21.15.57:0/3965643350 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.023+0000 7f920f400640 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a891e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 -- 172.21.15.57:0/3965643350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 msgr2=0x55705a891e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a891e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 -- 172.21.15.57:0/3965643350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 msgr2=0x55705a8b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a8b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 -- 172.21.15.57:0/3965643350 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705a88c5b0 con 0x55705a88fcd0 2024-09-12T03:14:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f920ea00640 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c6a0 tx=0x7f91fc0039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321ea06328edbc53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f91fb400640 1 -- 172.21.15.57:0/3965643350 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc09bbc0 con 0x55705a88fcd0 2024-09-12T03:14:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f91fb400640 1 -- 172.21.15.57:0/3965643350 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc0a4830 con 0x55705a88fcd0 2024-09-12T03:14:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.024+0000 7f91fb400640 1 -- 172.21.15.57:0/3965643350 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0ba2c0 con 0x55705a88fcd0 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- 172.21.15.57:0/3965643350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 msgr2=0x55705a8994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c6a0 tx=0x7f91fc0039b0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- 172.21.15.57:0/3965643350 shutdown_connections 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a8b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a8994f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 --2- 172.21.15.57:0/3965643350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a891e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- 172.21.15.57:0/3965643350 >> 172.21.15.57:0/3965643350 conn(0x55705a88b920 msgr2=0x55705a898de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- 172.21.15.57:0/3965643350 shutdown_connections 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- 172.21.15.57:0/3965643350 wait complete. 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.025+0000 7f9218e9d740 1 -- start start 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 Processor -- start 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a8a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a894400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f92101af6b0 con 0x55705a89a570 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f92101b0b20 con 0x55705a88fcd0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92101afd80 con 0x55705a88f3b0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f9218e9d740 10 client.0 _ll_register_callbacks cb 0x7f91fc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f920fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f920fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39398/0 (socket says 172.21.15.57:39398) 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f920fe00640 1 -- 172.21.15.57:0/4027997241 learned_addr learned my addr 172.21.15.57:0/4027997241 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f920ea00640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a894400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.026+0000 7f920f400640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a8a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 -- 172.21.15.57:0/4027997241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 msgr2=0x55705a8a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a8a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 -- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 msgr2=0x55705a894400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a894400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 -- 172.21.15.57:0/4027997241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705a895190 con 0x55705a89a570 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920ea00640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a894400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f920fe00640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9208004ab0 tx=0x7f9208003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b11c81d4cbd813 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f91f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208054020 con 0x55705a89a570 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f92080318c0 con 0x55705a89a570 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.027+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91fc0a4830 con 0x55705a89a570 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f920805f050 con 0x55705a89a570 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55705a914fd0 con 0x55705a89a570 2024-09-12T03:14:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91fc0a4830 con 0x55705a89a570 2024-09-12T03:14:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9208030070 con 0x55705a89a570 2024-09-12T03:14:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 20 client.15723 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a915300 0x55705a9176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 4 client.15723 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.028+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55705a7fcb80 con 0x55705a915300 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.029+0000 7f920f400640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a915300 0x55705a9176f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.029+0000 7f920f400640 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a915300 0x55705a9176f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8371655585a45d96 server_cookie=4376a620b74ff2b1 in_seq=0 out_seq=0 2024-09-12T03:14:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.029+0000 7f91f8c00640 10 client.15723 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.033+0000 7f91f8c00640 1 -- 172.21.15.57:0/4027997241 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55705a7fcb80 con 0x55705a915300 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.033+0000 7f91f8c00640 10 client.15723 handle_command_reply: tid=0 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.033+0000 7f9218e9d740 1 client.15723 shutdown 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 msgr2=0x55705a894960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9208004ab0 tx=0x7f9208003c60 comp rx=0 tx=0).stop 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 shutdown_connections 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55705a915300 0x55705a9176f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f91f8c00640 0 client.15723 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55705a89a570 0x55705a894960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55705a88fcd0 0x55705a894400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 --2- 172.21.15.57:0/4027997241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55705a88f3b0 0x55705a8a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 >> 172.21.15.57:0/4027997241 conn(0x55705a88b920 msgr2=0x55705a897df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 shutdown_connections 2024-09-12T03:14:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 1 -- 172.21.15.57:0/4027997241 wait complete. 2024-09-12T03:14:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.034+0000 7f9218e9d740 20 client.15723 trim_cache size 0 max 16384 2024-09-12T03:14:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:14:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:14:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:14:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 msgr2=0x7f91f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 0x7f91f40a5230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55705a88c780 tx=0x7f9200005f90 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 msgr2=0x7f921019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00e9f0 tx=0x7f91fc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 shutdown_connections 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f91f40a2d70 0x7f91f40a5230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f92101078c0 0x7f921019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210102250 0x7f921019a980 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 --2- 172.21.15.57:0/4134205408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92101018b0 0x7f921019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 >> 172.21.15.57:0/4134205408 conn(0x7f92100fd5d0 msgr2=0x7f921010d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 shutdown_connections 2024-09-12T03:14:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:32.035+0000 7f9214a00640 1 -- 172.21.15.57:0/4134205408 wait complete. 2024-09-12T03:14:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:31 smithi057 ceph-mon[30361]: pgmap v266: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:31 smithi204 ceph-mon[35013]: pgmap v266: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:33.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:33 smithi181 ceph-mon[35320]: pgmap v267: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:34.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:33 smithi057 ceph-mon[30361]: pgmap v267: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:34.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:33 smithi204 ceph-mon[35013]: pgmap v267: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:35.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:35 smithi181 ceph-mon[35320]: pgmap v268: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:14:36.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:35 smithi057 ceph-mon[30361]: pgmap v268: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:14:36.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:35 smithi204 ceph-mon[35013]: pgmap v268: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:14:37.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:37 smithi181 ceph-mon[35320]: pgmap v269: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:14:38.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:37 smithi057 ceph-mon[30361]: pgmap v269: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:14:38.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:37 smithi204 ceph-mon[35013]: pgmap v269: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:14:39.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:39 smithi181 ceph-mon[35320]: pgmap v270: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:40.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:39 smithi057 ceph-mon[30361]: pgmap v270: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:40.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:39 smithi204 ceph-mon[35013]: pgmap v270: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:41.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:41 smithi181 ceph-mon[35320]: pgmap v271: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:42.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:41 smithi057 ceph-mon[30361]: pgmap v271: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:42.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:41 smithi204 ceph-mon[35013]: pgmap v271: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:43.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:43 smithi181 ceph-mon[35320]: pgmap v272: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:44.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:43 smithi057 ceph-mon[30361]: pgmap v272: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:44.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:43 smithi204 ceph-mon[35013]: pgmap v272: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 -- 172.21.15.57:0/1075942587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0103b00 msgr2=0x7f44f0103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 --2- 172.21.15.57:0/1075942587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0103b00 0x7f44f0103f60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009930 tx=0x7f44dc02a440 comp rx=0 tx=0).stop 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 -- 172.21.15.57:0/1075942587 shutdown_connections 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 --2- 172.21.15.57:0/1075942587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f01044a0 0x7f44f01109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 --2- 172.21.15.57:0/1075942587 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0103b00 0x7f44f0103f60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 --2- 172.21.15.57:0/1075942587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f0109ba0 0x7f44f0109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 -- 172.21.15.57:0/1075942587 >> 172.21.15.57:0/1075942587 conn(0x7f44f00ff7c0 msgr2=0x7f44f0101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 -- 172.21.15.57:0/1075942587 shutdown_connections 2024-09-12T03:14:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.162+0000 7f44f6200640 1 -- 172.21.15.57:0/1075942587 wait complete. 2024-09-12T03:14:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.163+0000 7f44f6200640 1 Processor -- start 2024-09-12T03:14:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.163+0000 7f44f6200640 1 -- start start 2024-09-12T03:14:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f01044a0 0x7f44f019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 0x7f44f019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0113f60 con 0x7f44f0109ba0 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0113de0 con 0x7f44f0103b00 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01140e0 con 0x7f44f01044a0 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 0x7f44f019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f4e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37916/0 (socket says 172.21.15.57:37916) 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44f4e00640 1 -- 172.21.15.57:0/1197091895 learned_addr learned my addr 172.21.15.57:0/1197091895 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.164+0000 7f44efe00640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f01044a0 0x7f44f019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 -- 172.21.15.57:0/1197091895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f01044a0 msgr2=0x7f44f019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f01044a0 0x7f44f019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 -- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 msgr2=0x7f44f019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 0x7f44f019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 -- 172.21.15.57:0/1197091895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f019f460 con 0x7f44f0103b00 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f5800640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 0x7f44f019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f4e00640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9f0 tx=0x7f44e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44ed600640 1 -- 172.21.15.57:0/1197091895 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e400cd80 con 0x7f44f0103b00 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44ed600640 1 -- 172.21.15.57:0/1197091895 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e4010910 con 0x7f44f0103b00 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44ed600640 1 -- 172.21.15.57:0/1197091895 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4018990 con 0x7f44f0103b00 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f019f750 con 0x7f44f0103b00 2024-09-12T03:14:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.165+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a6fe0 con 0x7f44f0103b00 2024-09-12T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 Processor -- start 2024-09-12T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44ed600640 1 -- 172.21.15.57:0/1197091895 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f44e4010ac0 con 0x7f44f0103b00 2024-09-12T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 -- start start 2024-09-12T03:14:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5568ba974100 con 0x5568baa0e3b0 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5568ba91c1b0 con 0x5568baa19570 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44fa822740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568ba9700c0 con 0x5568baa0ecd0 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.169+0000 7f44f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44f4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37924/0 (socket says 172.21.15.57:37924) 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 -- 172.21.15.57:0/2206306358 learned_addr learned my addr 172.21.15.57:0/2206306358 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 -- 172.21.15.57:0/2206306358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 msgr2=0x5568baa19030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa19030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 -- 172.21.15.57:0/2206306358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 msgr2=0x5568baa0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 -- 172.21.15.57:0/2206306358 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568ba687210 con 0x5568baa19570 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44ed600640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 0x7f44d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44efe00640 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009930 tx=0x7f44dc005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=803087a063706348 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.170+0000 7f44ed600640 1 -- 172.21.15.57:0/1197091895 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f44e40cfb50 con 0x7f44f0103b00 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44f5800640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 0x7f44d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44e2000640 1 -- 172.21.15.57:0/2206306358 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc035040 con 0x5568baa19570 2024-09-12T03:14:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44e2000640 1 -- 172.21.15.57:0/2206306358 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc005520 con 0x5568baa19570 2024-09-12T03:14:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44e2000640 1 -- 172.21.15.57:0/2206306358 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0035f0 con 0x5568baa19570 2024-09-12T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 -- 172.21.15.57:0/2206306358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 msgr2=0x5568baa1b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009930 tx=0x7f44dc005de0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44f5800640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 0x7f44d80a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009fd0 tx=0x7f44e8009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 -- 172.21.15.57:0/2206306358 shutdown_connections 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa1b960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 --2- 172.21.15.57:0/2206306358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 -- 172.21.15.57:0/2206306358 >> 172.21.15.57:0/2206306358 conn(0x5568baa0a920 msgr2=0x5568baa0ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.171+0000 7f44fa822740 1 -- 172.21.15.57:0/2206306358 shutdown_connections 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.172+0000 7f44fa822740 1 -- 172.21.15.57:0/2206306358 wait complete. 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.172+0000 7f44fa822740 1 -- start start 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.172+0000 7f44fa822740 1 Processor -- start 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.172+0000 7f44fa822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa16570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 0x5568baa16ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5568baa1bff0 con 0x5568baa0ecd0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5568ba8fd090 con 0x5568baa0e3b0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568baa1c170 con 0x5568baa19570 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44fa822740 10 client.0 _ll_register_callbacks cb 0x5568ba951f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41982/0 (socket says 172.21.15.57:41982) 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 -- 172.21.15.57:0/2642960445 learned_addr learned my addr 172.21.15.57:0/2642960445 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa16570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44f5800640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 0x5568baa16ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 -- 172.21.15.57:0/2642960445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 msgr2=0x5568baa16ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 0x5568baa16ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 -- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 msgr2=0x5568baa16570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa16570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44efe00640 1 -- 172.21.15.57:0/2642960445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568ba753620 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.173+0000 7f44f5800640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 0x5568baa16ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44efe00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f44dc02a960 tx=0x7f44dc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e551adc2158a3d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc052060 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44dc005520 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f44dc004390 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44dc059050 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44fa822740 10 client.15735 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568baa27df0 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.174+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44dc005520 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f44dc0037b0 con 0x5568baa0ecd0 2024-09-12T03:14:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 10 client.15735 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 20 client.15735 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa29140 0x5568baa2b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 4 client.15735 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568ba981830 con 0x5568baa29140 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.175+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa29140 0x5568baa2b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.176+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa29140 0x5568baa2b530 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a54413ab9bd6f4f server_cookie=2d8ec36a39446cab in_seq=0 out_seq=0 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.176+0000 7f44d6a00640 10 client.15735 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44d6a00640 1 -- 172.21.15.57:0/2642960445 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568ba981830 con 0x5568baa29140 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44d6a00640 10 client.15735 handle_command_reply: tid=0 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 client.15735 shutdown 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 msgr2=0x5568baa16ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f44dc02a960 tx=0x7f44dc005b50 comp rx=0 tx=0).stop 2024-09-12T03:14:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 shutdown_connections 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa29140 0x5568baa2b530 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa19570 0x5568baa16ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0ecd0 0x5568baa16ab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44d6a00640 0 client.15735 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 --2- 172.21.15.57:0/2642960445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa16570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 >> 172.21.15.57:0/2642960445 conn(0x5568baa0a920 msgr2=0x5568baa10680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 shutdown_connections 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.177+0000 7f44fa822740 1 -- 172.21.15.57:0/2642960445 wait complete. 2024-09-12T03:14:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.178+0000 7f44fa822740 20 client.15735 trim_cache size 0 max 16384 2024-09-12T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 Processor -- start 2024-09-12T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 -- start start 2024-09-12T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b3b00 con 0x5568baa0e3b0 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b13d0 con 0x5568baa19570 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44fa822740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01af240 con 0x5568baa0ecd0 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.193+0000 7f44f4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44efe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41984/0 (socket says 172.21.15.57:41984) 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 -- 172.21.15.57:0/2751559745 learned_addr learned my addr 172.21.15.57:0/2751559745 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f5800640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 -- 172.21.15.57:0/2751559745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 msgr2=0x5568baa2ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 -- 172.21.15.57:0/2751559745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 msgr2=0x5568baa137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 -- 172.21.15.57:0/2751559745 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568baa1bcd0 con 0x5568baa0e3b0 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44efe00640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f5800640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44f4e00640 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f44e4098680 tx=0x7f44e40d2010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc71d1dd57889af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.194+0000 7f44e1600640 1 -- 172.21.15.57:0/2751559745 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e409b610 con 0x5568baa0e3b0 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44e1600640 1 -- 172.21.15.57:0/2751559745 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e409bcb0 con 0x5568baa0e3b0 2024-09-12T03:14:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44e1600640 1 -- 172.21.15.57:0/2751559745 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4018b30 con 0x5568baa0e3b0 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 -- 172.21.15.57:0/2751559745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 msgr2=0x5568baa2e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f44e4098680 tx=0x7f44e40d2010 comp rx=0 tx=0).stop 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 -- 172.21.15.57:0/2751559745 shutdown_connections 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa19570 0x5568baa137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 --2- 172.21.15.57:0/2751559745 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa0e3b0 0x5568baa2e550 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 -- 172.21.15.57:0/2751559745 >> 172.21.15.57:0/2751559745 conn(0x5568baa0a920 msgr2=0x5568baa184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 -- 172.21.15.57:0/2751559745 shutdown_connections 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.195+0000 7f44fa822740 1 -- 172.21.15.57:0/2751559745 wait complete. 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 -- start start 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 Processor -- start 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa2c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01af890 con 0x5568baa19570 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b0d00 con 0x5568baa0e3b0 2024-09-12T03:14:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01aff60 con 0x5568baa0ecd0 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.196+0000 7f44fa822740 10 client.0 _ll_register_callbacks cb 0x7f44e8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41998/0 (socket says 172.21.15.57:41998) 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 -- 172.21.15.57:0/3697063573 learned_addr learned my addr 172.21.15.57:0/3697063573 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44efe00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f4e00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa2c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 -- 172.21.15.57:0/3697063573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 msgr2=0x5568baa2c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 -- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 msgr2=0x5568baa2c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa2c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f5800640 1 -- 172.21.15.57:0/3697063573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568baa92b00 con 0x5568baa19570 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44efe00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.197+0000 7f44f4e00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa2c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44f5800640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44e8003f00 tx=0x7f44e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709c0440636742 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e800fe60 con 0x5568baa19570 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7f44e80047e0 con 0x5568baa19570 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e409bcb0 con 0x5568baa19570 2024-09-12T03:14:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44e802b440 con 0x5568baa19570 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44fa822740 10 client.15741 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568baa93e30 con 0x5568baa19570 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.198+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e409bcb0 con 0x5568baa19570 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f44e8029070 con 0x5568baa19570 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 10 client.15741 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 20 client.15741 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa94600 0x5568baa96a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 4 client.15741 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568ba97bb80 con 0x5568baa94600 2024-09-12T03:14:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.199+0000 7f44f4e00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa94600 0x5568baa96a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.200+0000 7f44f4e00640 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa94600 0x5568baa96a20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3edeaf7da02cf836 server_cookie=7d38fbe05b5e7916 in_seq=0 out_seq=0 2024-09-12T03:14:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.200+0000 7f44d6a00640 10 client.15741 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44d6a00640 1 -- 172.21.15.57:0/3697063573 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568ba97bb80 con 0x5568baa94600 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44d6a00640 10 client.15741 handle_command_reply: tid=0 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 client.15741 shutdown 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 msgr2=0x5568baa2cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f44e8003f00 tx=0x7f44e8004a40 comp rx=0 tx=0).stop 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 shutdown_connections 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5568baa94600 0x5568baa96a20 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5568baa19570 0x5568baa2cdb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568baa0ecd0 0x5568baa2c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 --2- 172.21.15.57:0/3697063573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5568baa0e3b0 0x5568baa2c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44d6a00640 0 client.15741 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.204+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 >> 172.21.15.57:0/3697063573 conn(0x5568baa0a920 msgr2=0x5568baa10630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 shutdown_connections 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44fa822740 1 -- 172.21.15.57:0/3697063573 wait complete. 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44fa822740 20 client.15741 trim_cache size 0 max 16384 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:14:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 msgr2=0x7f44d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 0x7f44d80a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009fd0 tx=0x7f44e8009500 comp rx=0 tx=0).stop 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 msgr2=0x7f44f019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9f0 tx=0x7f44e400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 shutdown_connections 2024-09-12T03:14:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f44d80a2cf0 0x7f44d80a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f44f0109ba0 0x7f44f019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44f01044a0 0x7f44f019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 --2- 172.21.15.57:0/1197091895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44f0103b00 0x7f44f019a470 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.205+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 >> 172.21.15.57:0/1197091895 conn(0x7f44f00ff7c0 msgr2=0x7f44f00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.206+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 shutdown_connections 2024-09-12T03:14:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:45.206+0000 7f44f6200640 1 -- 172.21.15.57:0/1197091895 wait complete. 2024-09-12T03:14:45.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:45 smithi181 ceph-mon[35320]: pgmap v273: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:46.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:45 smithi057 ceph-mon[30361]: pgmap v273: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:46.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:45 smithi204 ceph-mon[35013]: pgmap v273: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:47 smithi057 ceph-mon[30361]: pgmap v274: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:47 smithi204 ceph-mon[35013]: pgmap v274: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:47 smithi181 ceph-mon[35320]: pgmap v274: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:50.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:49 smithi057 ceph-mon[30361]: pgmap v275: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T03:14:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:49 smithi204 ceph-mon[35013]: pgmap v275: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T03:14:50.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:49 smithi181 ceph-mon[35320]: pgmap v275: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-12T03:14:52.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:51 smithi057 ceph-mon[30361]: pgmap v276: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:51 smithi204 ceph-mon[35013]: pgmap v276: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:52.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:51 smithi181 ceph-mon[35320]: pgmap v276: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:54.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:53 smithi057 ceph-mon[30361]: pgmap v277: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:54.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:53 smithi204 ceph-mon[35013]: pgmap v277: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:54.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:53 smithi181 ceph-mon[35320]: pgmap v277: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:14:56.005 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:55 smithi181 ceph-mon[35320]: pgmap v278: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:56.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:55 smithi057 ceph-mon[30361]: pgmap v278: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:55 smithi204 ceph-mon[35013]: pgmap v278: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:57 smithi057 ceph-mon[30361]: pgmap v279: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:57 smithi204 ceph-mon[35013]: pgmap v279: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:57 smithi181 ceph-mon[35320]: pgmap v279: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:14:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 -- 172.21.15.57:0/2520980620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1018b0 msgr2=0x7fdf2c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 --2- 172.21.15.57:0/2520980620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c101d10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009930 tx=0x7fdf1802a420 comp rx=0 tx=0).stop 2024-09-12T03:14:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 -- 172.21.15.57:0/2520980620 shutdown_connections 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 --2- 172.21.15.57:0/2520980620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 0x7fdf2c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 --2- 172.21.15.57:0/2520980620 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c101d10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 --2- 172.21.15.57:0/2520980620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 -- 172.21.15.57:0/2520980620 >> 172.21.15.57:0/2520980620 conn(0x7fdf2c0fd5d0 msgr2=0x7fdf2c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 -- 172.21.15.57:0/2520980620 shutdown_connections 2024-09-12T03:14:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.332+0000 7fdf32200640 1 -- 172.21.15.57:0/2520980620 wait complete. 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.333+0000 7fdf32200640 1 Processor -- start 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.333+0000 7fdf32200640 1 -- start start 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 0x7fdf2c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111d50 con 0x7fdf2c1078c0 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111bd0 con 0x7fdf2c1018b0 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf32200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111ed0 con 0x7fdf2c102250 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf31800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.336 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf31800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41328/0 (socket says 172.21.15.57:41328) 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf31800640 1 -- 172.21.15.57:0/2338400728 learned_addr learned my addr 172.21.15.57:0/2338400728 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.334+0000 7fdf2be00640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 0x7fdf2c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 -- 172.21.15.57:0/2338400728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 msgr2=0x7fdf2c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 0x7fdf2c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 -- 172.21.15.57:0/2338400728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 msgr2=0x7fdf2c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 -- 172.21.15.57:0/2338400728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c19f420 con 0x7fdf2c1078c0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf30e00640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf31800640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e970 tx=0x7fdf2400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf29600640 1 -- 172.21.15.57:0/2338400728 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400ccc0 con 0x7fdf2c1078c0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf29600640 1 -- 172.21.15.57:0/2338400728 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2400ce60 con 0x7fdf2c1078c0 2024-09-12T03:14:58.337 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c19f710 con 0x7fdf2c1078c0 2024-09-12T03:14:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf29600640 1 -- 172.21.15.57:0/2338400728 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24010490 con 0x7fdf2c1078c0 2024-09-12T03:14:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.335+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a6fa0 con 0x7fdf2c1078c0 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.338+0000 7fdf36640740 1 Processor -- start 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.338+0000 7fdf29600640 1 -- 172.21.15.57:0/2338400728 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fdf24005020 con 0x7fdf2c1078c0 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 -- start start 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61a3b0 0x55a31b61a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b627960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a31b580100 con 0x55a31b61acd0 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a31b5281b0 con 0x55a31b61a3b0 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf36640740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a31b57c0c0 con 0x55a31b625570 2024-09-12T03:14:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41330/0 (socket says 172.21.15.57:41330) 2024-09-12T03:14:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf30e00640 1 -- 172.21.15.57:0/2694029318 learned_addr learned my addr 172.21.15.57:0/2694029318 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61a3b0 0x55a31b61a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.339+0000 7fdf2be00640 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b627960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 -- 172.21.15.57:0/2694029318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 msgr2=0x55a31b627960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b627960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 -- 172.21.15.57:0/2694029318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61a3b0 msgr2=0x55a31b61a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61a3b0 0x55a31b61a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 -- 172.21.15.57:0/2694029318 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31b293210 con 0x55a31b61acd0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf29600640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 0x7fdf100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf30e00640 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f40 tx=0x7fdf1c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e70d269a8642d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf29600640 1 -- 172.21.15.57:0/2338400728 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7fdf240cfce0 con 0x7fdf2c1078c0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.340+0000 7fdf2be00640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 0x7fdf100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf22000640 1 -- 172.21.15.57:0/2694029318 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c011220 con 0x55a31b61acd0 2024-09-12T03:14:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf22000640 1 -- 172.21.15.57:0/2694029318 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c00f9a0 con 0x55a31b61acd0 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf22000640 1 -- 172.21.15.57:0/2694029318 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c004580 con 0x55a31b61acd0 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf36640740 1 -- 172.21.15.57:0/2694029318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 msgr2=0x55a31b625030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf36640740 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f40 tx=0x7fdf1c011cb0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.341+0000 7fdf2be00640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 0x7fdf100a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf18006010 tx=0x7fdf1802a420 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 -- 172.21.15.57:0/2694029318 shutdown_connections 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b627960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b625030 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 --2- 172.21.15.57:0/2694029318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61a3b0 0x55a31b61a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 -- 172.21.15.57:0/2694029318 >> 172.21.15.57:0/2694029318 conn(0x55a31b616920 msgr2=0x55a31b616d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 -- 172.21.15.57:0/2694029318 shutdown_connections 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 -- 172.21.15.57:0/2694029318 wait complete. 2024-09-12T03:14:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 -- start start 2024-09-12T03:14:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.342+0000 7fdf36640740 1 Processor -- start 2024-09-12T03:14:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61acd0 0x55a31b622c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b623150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a31b627ff0 con 0x55a31b61a3b0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a31b509090 con 0x55a31b61acd0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a31b628170 con 0x55a31b625570 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf36640740 10 client.0 _ll_register_callbacks cb 0x55a31b55df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41334/0 (socket says 172.21.15.57:41334) 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf30e00640 1 -- 172.21.15.57:0/1209977474 learned_addr learned my addr 172.21.15.57:0/1209977474 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61acd0 0x55a31b622c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.343+0000 7fdf31800640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b623150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 -- 172.21.15.57:0/1209977474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 msgr2=0x55a31b623150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b623150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 -- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61acd0 msgr2=0x55a31b622c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61acd0 0x55a31b622c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 -- 172.21.15.57:0/1209977474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31b35f620 con 0x55a31b61a3b0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f10 tx=0x7fdf1c003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1ac39531d42353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf16a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c01e040 con 0x55a31b61a3b0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7fdf1c004c40 con 0x55a31b61a3b0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c00f9a0 con 0x55a31b61a3b0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf1c031050 con 0x55a31b61a3b0 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf36640740 10 client.15753 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.344+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a31b633e10 con 0x55a31b61a3b0 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c00f9a0 con 0x55a31b61a3b0 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf1c018070 con 0x55a31b61a3b0 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 10 client.15753 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 20 client.15753 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b635270 0x55a31b637660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 4 client.15753 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.345+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a31b58d830 con 0x55a31b635270 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.346+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b635270 0x55a31b637660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.346+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b635270 0x55a31b637660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb04c66800f3d58 server_cookie=bc6d74e17a75fc27 in_seq=0 out_seq=0 2024-09-12T03:14:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.346+0000 7fdf16a00640 10 client.15753 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf16a00640 1 -- 172.21.15.57:0/1209977474 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a31b58d830 con 0x55a31b635270 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf16a00640 10 client.15753 handle_command_reply: tid=0 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 client.15753 shutdown 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 msgr2=0x55a31b6226d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f10 tx=0x7fdf1c003c60 comp rx=0 tx=0).stop 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 shutdown_connections 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b635270 0x55a31b637660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b625570 0x55a31b623150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b61acd0 0x55a31b622c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 --2- 172.21.15.57:0/1209977474 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b6226d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 >> 172.21.15.57:0/1209977474 conn(0x55a31b616920 msgr2=0x55a31b61c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf16a00640 0 client.15753 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 shutdown_connections 2024-09-12T03:14:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.347+0000 7fdf36640740 1 -- 172.21.15.57:0/1209977474 wait complete. 2024-09-12T03:14:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.348+0000 7fdf36640740 20 client.15753 trim_cache size 0 max 16384 2024-09-12T03:14:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 Processor -- start 2024-09-12T03:14:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 -- start start 2024-09-12T03:14:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61acd0 0x55a31b63d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b623e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b3b80 con 0x55a31b61a3b0 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b1450 con 0x55a31b625570 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.363+0000 7fdf36640740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1af2c0 con 0x55a31b61acd0 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41338/0 (socket says 172.21.15.57:41338) 2024-09-12T03:14:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 -- 172.21.15.57:0/1624405184 learned_addr learned my addr 172.21.15.57:0/1624405184 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf31800640 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b623e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61acd0 0x55a31b63d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 -- 172.21.15.57:0/1624405184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61acd0 msgr2=0x55a31b63d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61acd0 0x55a31b63d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 -- 172.21.15.57:0/1624405184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 msgr2=0x55a31b623e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b623e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 -- 172.21.15.57:0/1624405184 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a31b61a130 con 0x55a31b61a3b0 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f40 tx=0x7fdf1c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b907a66e66cddf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.364+0000 7fdf21600640 1 -- 172.21.15.57:0/1624405184 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00fe60 con 0x55a31b61a3b0 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf21600640 1 -- 172.21.15.57:0/1624405184 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c028070 con 0x55a31b61a3b0 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf21600640 1 -- 172.21.15.57:0/1624405184 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0059c0 con 0x55a31b61a3b0 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 -- 172.21.15.57:0/1624405184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 msgr2=0x55a31b63cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c005f40 tx=0x7fdf1c009f00 comp rx=0 tx=0).stop 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 -- 172.21.15.57:0/1624405184 shutdown_connections 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b623e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61acd0 0x55a31b63d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 --2- 172.21.15.57:0/1624405184 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61a3b0 0x55a31b63cda0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 -- 172.21.15.57:0/1624405184 >> 172.21.15.57:0/1624405184 conn(0x55a31b616920 msgr2=0x55a31b620fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 -- 172.21.15.57:0/1624405184 shutdown_connections 2024-09-12T03:14:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.365+0000 7fdf36640740 1 -- 172.21.15.57:0/1624405184 wait complete. 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 -- start start 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 Processor -- start 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61a3b0 0x55a31b620050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b62d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1af910 con 0x55a31b61acd0 2024-09-12T03:14:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1b0d80 con 0x55a31b625570 2024-09-12T03:14:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c1affe0 con 0x55a31b61a3b0 2024-09-12T03:14:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf36640740 10 client.0 _ll_register_callbacks cb 0x7fdf24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:14:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.366+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:41350/0 (socket says 172.21.15.57:41350) 2024-09-12T03:14:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 -- 172.21.15.57:0/1928376537 learned_addr learned my addr 172.21.15.57:0/1928376537 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61a3b0 0x55a31b620050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf31800640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b62d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 -- 172.21.15.57:0/1928376537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61a3b0 msgr2=0x55a31b620050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61a3b0 0x55a31b620050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 -- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 msgr2=0x55a31b62d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:14:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b62d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 -- 172.21.15.57:0/1928376537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a31b62dec0 con 0x55a31b61acd0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf2be00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802aa10 tx=0x7fdf18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe033bd894932e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf16a00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.367+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1805b060 con 0x55a31b61acd0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c028070 con 0x55a31b61acd0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9554+0+0 (secure 0 0 0) 0x7fdf180046c0 con 0x55a31b61acd0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf18062050 con 0x55a31b61acd0 2024-09-12T03:14:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 10 client.15759 fetch_fsmap learned FSMap version 30 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a31b69ed70 con 0x55a31b61acd0 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c028070 con 0x55a31b61acd0 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf18049040 con 0x55a31b61acd0 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 10 client.15759 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 20 client.15759 populate_metadata read hostname 'smithi057' 2024-09-12T03:14:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b69f470 0x55a31b6a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 4 client.15759 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:14:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.368+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a31b587b80 con 0x55a31b69f470 2024-09-12T03:14:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.369+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b69f470 0x55a31b6a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.369+0000 7fdf30e00640 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b69f470 0x55a31b6a1890 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225e7873c6f29270 server_cookie=71c14322c73a0e0c in_seq=0 out_seq=0 2024-09-12T03:14:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.369+0000 7fdf16a00640 10 client.15759 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf16a00640 1 -- 172.21.15.57:0/1928376537 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a31b587b80 con 0x55a31b69f470 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf16a00640 10 client.15759 handle_command_reply: tid=0 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 client.15759 shutdown 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 msgr2=0x55a31b62d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf1802aa10 tx=0x7fdf18000bc0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 shutdown_connections 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a31b69f470 0x55a31b6a1890 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a31b625570 0x55a31b62d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a31b61acd0 0x55a31b62d3c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 --2- 172.21.15.57:0/1928376537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a31b61a3b0 0x55a31b620050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf16a00640 0 client.15759 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 >> 172.21.15.57:0/1928376537 conn(0x55a31b616920 msgr2=0x55a31b6159c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 shutdown_connections 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 1 -- 172.21.15.57:0/1928376537 wait complete. 2024-09-12T03:14:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.374+0000 7fdf36640740 20 client.15759 trim_cache size 0 max 16384 2024-09-12T03:14:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:14:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:14:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:14:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:14:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 msgr2=0x7fdf100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 0x7fdf100a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf18006010 tx=0x7fdf1802a420 comp rx=0 tx=0).stop 2024-09-12T03:14:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 msgr2=0x7fdf2c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e970 tx=0x7fdf2400ed70 comp rx=0 tx=0).stop 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 shutdown_connections 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fdf100a2cf0 0x7fdf100a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fdf2c1078c0 0x7fdf2c19ecf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf2c102250 0x7fdf2c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 --2- 172.21.15.57:0/2338400728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf2c1018b0 0x7fdf2c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 >> 172.21.15.57:0/2338400728 conn(0x7fdf2c0fd5d0 msgr2=0x7fdf2c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 shutdown_connections 2024-09-12T03:14:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:14:58.375+0000 7fdf32200640 1 -- 172.21.15.57:0/2338400728 wait complete. 2024-09-12T03:15:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:14:59 smithi057 ceph-mon[30361]: pgmap v280: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:00.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:14:59 smithi204 ceph-mon[35013]: pgmap v280: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:14:59 smithi181 ceph-mon[35320]: pgmap v280: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:01 smithi057 ceph-mon[30361]: pgmap v281: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:15:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:01 smithi204 ceph-mon[35013]: pgmap v281: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:15:02.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:01 smithi181 ceph-mon[35320]: pgmap v281: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:15:04.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:03 smithi057 ceph-mon[30361]: pgmap v282: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:15:04.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:03 smithi204 ceph-mon[35013]: pgmap v282: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:15:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:03 smithi181 ceph-mon[35320]: pgmap v282: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 7.4 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:15:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:05 smithi057 ceph-mon[30361]: pgmap v283: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:15:06.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:05 smithi204 ceph-mon[35013]: pgmap v283: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:15:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:05 smithi181 ceph-mon[35320]: pgmap v283: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:15:08.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:07 smithi057 ceph-mon[30361]: pgmap v284: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:15:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:07 smithi204 ceph-mon[35013]: pgmap v284: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:15:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:07 smithi181 ceph-mon[35320]: pgmap v284: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 22 op/s 2024-09-12T03:15:09.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:08 smithi057 ceph-mon[30361]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:15:09.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:08 smithi204 ceph-mon[35013]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:15:09.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:08 smithi181 ceph-mon[35320]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T03:15:10.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:09 smithi057 ceph-mon[30361]: pgmap v286: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:10.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:09 smithi204 ceph-mon[35013]: pgmap v286: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:10.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:09 smithi181 ceph-mon[35320]: pgmap v286: 129 pgs: 129 active+clean; 36 GiB data, 76 GiB used, 997 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 -- 172.21.15.57:0/4228524391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810109b80 msgr2=0x7fa810109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 --2- 172.21.15.57:0/4228524391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810109b80 0x7fa810109f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa804009930 tx=0x7fa80402a420 comp rx=0 tx=0).stop 2024-09-12T03:15:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 -- 172.21.15.57:0/4228524391 shutdown_connections 2024-09-12T03:15:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 --2- 172.21.15.57:0/4228524391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810104480 0x7fa8101109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 --2- 172.21.15.57:0/4228524391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa810103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 --2- 172.21.15.57:0/4228524391 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810109b80 0x7fa810109f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 -- 172.21.15.57:0/4228524391 >> 172.21.15.57:0/4228524391 conn(0x7fa8100ff7c0 msgr2=0x7fa810101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 -- 172.21.15.57:0/4228524391 shutdown_connections 2024-09-12T03:15:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.514+0000 7fa817200640 1 -- 172.21.15.57:0/4228524391 wait complete. 2024-09-12T03:15:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.515+0000 7fa817200640 1 Processor -- start 2024-09-12T03:15:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.515+0000 7fa817200640 1 -- start start 2024-09-12T03:15:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa81019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810109b80 0x7fa81019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa810113f40 con 0x7fa810104480 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa810113dc0 con 0x7fa810103ae0 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa817200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101140c0 con 0x7fa810109b80 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44170/0 (socket says 172.21.15.57:44170) 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815400640 1 -- 172.21.15.57:0/845409797 learned_addr learned my addr 172.21.15.57:0/845409797 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa816800640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810109b80 0x7fa81019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815e00640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa81019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815400640 1 -- 172.21.15.57:0/845409797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810109b80 msgr2=0x7fa81019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.516+0000 7fa815400640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810109b80 0x7fa81019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa815400640 1 -- 172.21.15.57:0/845409797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 msgr2=0x7fa81019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa815400640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa81019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa815400640 1 -- 172.21.15.57:0/845409797 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81019f4c0 con 0x7fa810104480 2024-09-12T03:15:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa815e00640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa81019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa815400640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e9f0 tx=0x7fa80000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa80aa00640 1 -- 172.21.15.57:0/845409797 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80000cd80 con 0x7fa810104480 2024-09-12T03:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa80aa00640 1 -- 172.21.15.57:0/845409797 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa800010910 con 0x7fa810104480 2024-09-12T03:15:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa80aa00640 1 -- 172.21.15.57:0/845409797 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800018990 con 0x7fa810104480 2024-09-12T03:15:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81019f7b0 con 0x7fa810104480 2024-09-12T03:15:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.517+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101a7040 con 0x7fa810104480 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.520+0000 7fa81b6c8740 1 Processor -- start 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.520+0000 7fa80aa00640 1 -- 172.21.15.57:0/845409797 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fa800010ac0 con 0x7fa810104480 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 -- start start 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aad7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56164aa3d100 con 0x56164aae2570 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56164a9e51b0 con 0x56164aad7cd0 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56164aa390c0 con 0x56164aad73b0 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa815400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa815e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa815e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:52778/0 (socket says 172.21.15.57:52778) 2024-09-12T03:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa815400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44172/0 (socket says 172.21.15.57:44172) 2024-09-12T03:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa816800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aad7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.521+0000 7fa815e00640 1 -- 172.21.15.57:0/3774370030 learned_addr learned my addr 172.21.15.57:0/3774370030 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 -- 172.21.15.57:0/3774370030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 msgr2=0x56164aad7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aad7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 -- 172.21.15.57:0/3774370030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 msgr2=0x56164aae4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 -- 172.21.15.57:0/3774370030 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164a750210 con 0x56164aad7cd0 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa815e00640 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa804000c00 tx=0x7fa8040040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcbc34789d4488e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa80aa00640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa7ff000640 1 -- 172.21.15.57:0/3774370030 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804035030 con 0x56164aad7cd0 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa7ff000640 1 -- 172.21.15.57:0/3774370030 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804005520 con 0x56164aad7cd0 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.522+0000 7fa7ff000640 1 -- 172.21.15.57:0/3774370030 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8040035f0 con 0x56164aad7cd0 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa80aa00640 1 -- 172.21.15.57:0/845409797 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa8000cfc30 con 0x7fa810104480 2024-09-12T03:15:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa816800640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3774370030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 msgr2=0x56164aae2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa804000c00 tx=0x7fa8040040c0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3774370030 shutdown_connections 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aae2030 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3774370030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aad7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3774370030 >> 172.21.15.57:0/3774370030 conn(0x56164aad3920 msgr2=0x56164aad3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa816800640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 0x7fa7f80a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005fd0 tx=0x7fa80c009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3774370030 shutdown_connections 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.523+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3774370030 wait complete. 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.524+0000 7fa81b6c8740 1 -- start start 2024-09-12T03:15:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.524+0000 7fa81b6c8740 1 Processor -- start 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.524+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.524+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad7cd0 0x56164aadfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.524+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x56164aae4ff0 con 0x56164aae2570 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x56164a9c6090 con 0x56164aad73b0 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56164aae5170 con 0x56164aad7cd0 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa81b6c8740 10 client.0 _ll_register_callbacks cb 0x56164aa1af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:52790/0 (socket says 172.21.15.57:52790) 2024-09-12T03:15:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 -- 172.21.15.57:0/2866055870 learned_addr learned my addr 172.21.15.57:0/2866055870 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa816800640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815400640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad7cd0 0x56164aadfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 -- 172.21.15.57:0/2866055870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad7cd0 msgr2=0x56164aadfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad7cd0 0x56164aadfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 -- 172.21.15.57:0/2866055870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 msgr2=0x56164aae0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 -- 172.21.15.57:0/2866055870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164a81c620 con 0x56164aad73b0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa815e00640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa80402aa10 tx=0x7fa8040045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab054c02a56ffcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.525+0000 7fa7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804035070 con 0x56164aad73b0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa804005520 con 0x56164aad73b0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa804004ef0 con 0x56164aad73b0 2024-09-12T03:15:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8040050f0 con 0x56164aad73b0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa81b6c8740 10 client.25916 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56164aaf0df0 con 0x56164aad73b0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804005520 con 0x56164aad73b0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa8040036f0 con 0x56164aad73b0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa81b6c8740 10 client.25916 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.526+0000 7fa81b6c8740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa81b6c8740 20 client.25916 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164aaf2140 0x56164aaf4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa81b6c8740 4 client.25916 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164aa4a830 con 0x56164aaf2140 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa815400640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164aaf2140 0x56164aaf4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa815400640 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164aaf2140 0x56164aaf4530 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8108191971315c1f server_cookie=ebb1cb099c4e5125 in_seq=0 out_seq=0 2024-09-12T03:15:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.527+0000 7fa7f3400640 10 client.25916 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa7f3400640 1 -- 172.21.15.57:0/2866055870 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56164aa4a830 con 0x56164aaf2140 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa7f3400640 10 client.25916 handle_command_reply: tid=0 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 client.25916 shutdown 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 msgr2=0x56164aadf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa80402aa10 tx=0x7fa8040045a0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 shutdown_connections 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164aaf2140 0x56164aaf4530 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aae0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad7cd0 0x56164aadfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/2866055870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad73b0 0x56164aadf6c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 >> 172.21.15.57:0/2866055870 conn(0x56164aad3920 msgr2=0x56164aad9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa7f3400640 0 client.25916 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 shutdown_connections 2024-09-12T03:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.528+0000 7fa81b6c8740 1 -- 172.21.15.57:0/2866055870 wait complete. 2024-09-12T03:15:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.529+0000 7fa81b6c8740 20 client.25916 trim_cache size 0 max 16384 2024-09-12T03:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 Processor -- start 2024-09-12T03:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 -- start start 2024-09-12T03:15:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aadfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aaf9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b3ca0 con 0x56164aae2570 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b1570 con 0x56164aad7cd0 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101af3e0 con 0x56164aad73b0 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa815400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa815400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:52796/0 (socket says 172.21.15.57:52796) 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa815400640 1 -- 172.21.15.57:0/28029816 learned_addr learned my addr 172.21.15.57:0/28029816 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa816800640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aaf9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.544+0000 7fa815e00640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aadfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 -- 172.21.15.57:0/28029816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 msgr2=0x56164aadfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aadfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 -- 172.21.15.57:0/28029816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 msgr2=0x56164aaf9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aaf9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 -- 172.21.15.57:0/28029816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164aade670 con 0x56164aad7cd0 2024-09-12T03:15:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa816800640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aaf9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa815400640 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa800098660 tx=0x7fa8000d2080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3001be34c4d6cedd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa7fe600640 1 -- 172.21.15.57:0/28029816 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80009b7d0 con 0x56164aad7cd0 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa7fe600640 1 -- 172.21.15.57:0/28029816 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80009c390 con 0x56164aad7cd0 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.545+0000 7fa7fe600640 1 -- 172.21.15.57:0/28029816 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80009b5f0 con 0x56164aad7cd0 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- 172.21.15.57:0/28029816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 msgr2=0x56164aad9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa800098660 tx=0x7fa8000d2080 comp rx=0 tx=0).stop 2024-09-12T03:15:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- 172.21.15.57:0/28029816 shutdown_connections 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aae2570 0x56164aaf9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aad9e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/28029816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aad73b0 0x56164aadfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- 172.21.15.57:0/28029816 >> 172.21.15.57:0/28029816 conn(0x56164aad3920 msgr2=0x56164aace6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- 172.21.15.57:0/28029816 shutdown_connections 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- 172.21.15.57:0/28029816 wait complete. 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.546+0000 7fa81b6c8740 1 -- start start 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 Processor -- start 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aaea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aae2570 0x56164aae7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101afa30 con 0x56164aad73b0 2024-09-12T03:15:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b0ea0 con 0x56164aad7cd0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b0100 con 0x56164aae2570 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa81b6c8740 10 client.0 _ll_register_callbacks cb 0x7fa8000d35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa815e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa815e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44192/0 (socket says 172.21.15.57:44192) 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa815e00640 1 -- 172.21.15.57:0/3518731109 learned_addr learned my addr 172.21.15.57:0/3518731109 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa816800640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aae2570 0x56164aae7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.547+0000 7fa815400640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aaea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 -- 172.21.15.57:0/3518731109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aae2570 msgr2=0x56164aae7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aae2570 0x56164aae7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 -- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 msgr2=0x56164aaea900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aaea900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 -- 172.21.15.57:0/3518731109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164aaf48a0 con 0x56164aad73b0 2024-09-12T03:15:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa815e00640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa804031550 tx=0x7fa80402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ee37bbdb641b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804035040 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9642+0+0 (secure 0 0 0) 0x7fa804004ef0 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80009c390 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.548+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8040049d0 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 10 client.15777 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56164ab5d000 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80009c390 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa804059060 con 0x56164aad73b0 2024-09-12T03:15:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 10 client.15777 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 20 client.15777 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164ab5d6c0 0x56164ab5fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 4 client.15777 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56164aa44b80 con 0x56164ab5d6c0 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.549+0000 7fa815400640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164ab5d6c0 0x56164ab5fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.550+0000 7fa815400640 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164ab5d6c0 0x56164ab5fab0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404e52faab1d41 server_cookie=9fe1d671480680da in_seq=0 out_seq=0 2024-09-12T03:15:11.554 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.550+0000 7fa7f3400640 10 client.15777 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.556+0000 7fa7f3400640 1 -- 172.21.15.57:0/3518731109 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56164aa44b80 con 0x56164ab5d6c0 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.556+0000 7fa7f3400640 10 client.15777 handle_command_reply: tid=0 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.556+0000 7fa81b6c8740 1 client.15777 shutdown 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.556+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 msgr2=0x56164aaea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.556+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa804031550 tx=0x7fa80402a420 comp rx=0 tx=0).stop 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 shutdown_connections 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x56164ab5d6c0 0x56164ab5fab0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56164aae2570 0x56164aae7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56164aad7cd0 0x56164aaea900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 --2- 172.21.15.57:0/3518731109 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x56164aad73b0 0x56164aaea3c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa7f3400640 0 client.15777 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 >> 172.21.15.57:0/3518731109 conn(0x56164aad3920 msgr2=0x56164aad29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 shutdown_connections 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 1 -- 172.21.15.57:0/3518731109 wait complete. 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa81b6c8740 20 client.15777 trim_cache size 0 max 16384 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:15:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 msgr2=0x7fa7f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 0x7fa7f80a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005fd0 tx=0x7fa80c009500 comp rx=0 tx=0).stop 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 msgr2=0x7fa81019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.557+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e9f0 tx=0x7fa80000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 shutdown_connections 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa810109b80 0x7fa81019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7f80a2d70 0x7fa7f80a5230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa810104480 0x7fa81019aa10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 --2- 172.21.15.57:0/845409797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa810103ae0 0x7fa81019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 >> 172.21.15.57:0/845409797 conn(0x7fa8100ff7c0 msgr2=0x7fa8101000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 shutdown_connections 2024-09-12T03:15:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:11.558+0000 7fa817200640 1 -- 172.21.15.57:0/845409797 wait complete. 2024-09-12T03:15:12.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:11 smithi057 ceph-mon[30361]: pgmap v287: 129 pgs: 3 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 118 active+clean; 35 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:12.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:11 smithi204 ceph-mon[35013]: pgmap v287: 129 pgs: 3 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 118 active+clean; 35 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:12.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:11 smithi181 ceph-mon[35320]: pgmap v287: 129 pgs: 3 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 118 active+clean; 35 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 18 KiB/s rd, 56 KiB/s wr, 27 op/s 2024-09-12T03:15:14.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:13 smithi057 ceph-mon[30361]: pgmap v288: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:14.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:13 smithi204 ceph-mon[35013]: pgmap v288: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:13 smithi181 ceph-mon[35320]: pgmap v288: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:16.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:15 smithi057 ceph-mon[30361]: pgmap v289: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:16.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:15 smithi204 ceph-mon[35013]: pgmap v289: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:16.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:15 smithi181 ceph-mon[35320]: pgmap v289: 129 pgs: 6 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 110 active+clean; 27 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 16 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:15:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:17 smithi057 ceph-mon[30361]: pgmap v290: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T03:15:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:17 smithi204 ceph-mon[35013]: pgmap v290: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T03:15:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:17 smithi181 ceph-mon[35320]: pgmap v290: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T03:15:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:19 smithi057 ceph-mon[30361]: pgmap v291: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-09-12T03:15:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:19 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:19 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:19 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:20.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:19 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:15:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:19 smithi204 ceph-mon[35013]: pgmap v291: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-09-12T03:15:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:19 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:19 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:19 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:19 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:15:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:19 smithi181 ceph-mon[35320]: pgmap v291: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.1 KiB/s wr, 6 op/s 2024-09-12T03:15:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:19 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:19 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:19 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:19 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:15:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:21 smithi057 ceph-mon[30361]: pgmap v292: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-12T03:15:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:21 smithi057 ceph-mon[30361]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:15:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:21 smithi204 ceph-mon[35013]: pgmap v292: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-12T03:15:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:21 smithi204 ceph-mon[35013]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:15:22.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:21 smithi181 ceph-mon[35320]: pgmap v292: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-12T03:15:22.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:21 smithi181 ceph-mon[35320]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T03:15:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:23 smithi057 ceph-mon[30361]: pgmap v294: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:24.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:23 smithi204 ceph-mon[35013]: pgmap v294: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:23 smithi181 ceph-mon[35320]: pgmap v294: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 -- 172.21.15.57:0/150669255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8104900 msgr2=0x7f2dc8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 --2- 172.21.15.57:0/150669255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8104900 0x7f2dc8108940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009900 tx=0x7f2db402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 -- 172.21.15.57:0/150669255 shutdown_connections 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 --2- 172.21.15.57:0/150669255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc81090a0 0x7f2dc8110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 --2- 172.21.15.57:0/150669255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8104900 0x7f2dc8108940 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 --2- 172.21.15.57:0/150669255 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc8103f50 0x7f2dc8104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.685+0000 7f2dcee00640 1 -- 172.21.15.57:0/150669255 >> 172.21.15.57:0/150669255 conn(0x7f2dc80ff9e0 msgr2=0x7f2dc8101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.686+0000 7f2dcee00640 1 -- 172.21.15.57:0/150669255 shutdown_connections 2024-09-12T03:15:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.686+0000 7f2dcee00640 1 -- 172.21.15.57:0/150669255 wait complete. 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.686+0000 7f2dcee00640 1 Processor -- start 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 -- start start 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc81090a0 0x7f2dc819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc819ef60 0x7f2dc819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81140f0 con 0x7f2dc81090a0 2024-09-12T03:15:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8113f70 con 0x7f2dc8103f50 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8114270 con 0x7f2dc819ef60 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dc7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dc7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc81090a0 0x7f2dc819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dc7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41948/0 (socket says 172.21.15.57:41948) 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dc7e00640 1 -- 172.21.15.57:0/3305718733 learned_addr learned my addr 172.21.15.57:0/3305718733 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.687+0000 7f2dcce00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc819ef60 0x7f2dc819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 -- 172.21.15.57:0/3305718733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc819ef60 msgr2=0x7f2dc819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc819ef60 0x7f2dc819f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 -- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc81090a0 msgr2=0x7f2dc819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc81090a0 0x7f2dc819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 -- 172.21.15.57:0/3305718733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc819fa50 con 0x7f2dc8103f50 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc7e00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2db800ea60 tx=0x7f2db800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc4c00640 1 -- 172.21.15.57:0/3305718733 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db800cd20 con 0x7f2dc8103f50 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc4c00640 1 -- 172.21.15.57:0/3305718733 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db80035f0 con 0x7f2dc8103f50 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dc4c00640 1 -- 172.21.15.57:0/3305718733 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8010690 con 0x7f2dc8103f50 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.688+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc819fd40 con 0x7f2dc8103f50 2024-09-12T03:15:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.689+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc81a74b0 con 0x7f2dc8103f50 2024-09-12T03:15:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.691+0000 7f2dd1c5d740 1 Processor -- start 2024-09-12T03:15:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 -- start start 2024-09-12T03:15:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dc4c00640 1 -- 172.21.15.57:0/3305718733 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f2db80037a0 con 0x7f2dc8103f50 2024-09-12T03:15:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e13912790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560e1386b100 con 0x560e139051e0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e138131b0 con 0x560e13905b00 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e138670c0 con 0x560e139103a0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59002/0 (socket says 172.21.15.57:59002) 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dc7e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dcce00640 1 -- 172.21.15.57:0/2053949590 learned_addr learned my addr 172.21.15.57:0/2053949590 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.692+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e13912790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 -- 172.21.15.57:0/2053949590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 msgr2=0x560e13912790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e13912790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 -- 172.21.15.57:0/2053949590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 msgr2=0x560e1390fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 -- 172.21.15.57:0/2053949590 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1357e210 con 0x560e139051e0 2024-09-12T03:15:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e13912790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009f60 tx=0x7f2dc0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f750bf194f2519b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dc4c00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 0x7f2dac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dbd600640 1 -- 172.21.15.57:0/2053949590 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc001e070 con 0x560e139051e0 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.693+0000 7f2dbd600640 1 -- 172.21.15.57:0/2053949590 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0003c30 con 0x560e139051e0 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dbd600640 1 -- 172.21.15.57:0/2053949590 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000f9c0 con 0x560e139051e0 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dc4c00640 1 -- 172.21.15.57:0/3305718733 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f2db80cfb40 con 0x7f2dc8103f50 2024-09-12T03:15:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dc7e00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 0x7f2dac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2053949590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 msgr2=0x560e139055c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009f60 tx=0x7f2dc0009f90 comp rx=0 tx=0).stop 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dc7e00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 0x7f2dac0a52e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x560e139025b0 tx=0x7f2db80d35f0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2053949590 shutdown_connections 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e13912790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2053949590 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e139055c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.694+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2053949590 >> 172.21.15.57:0/2053949590 conn(0x560e13901750 msgr2=0x560e13901b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.695+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2053949590 shutdown_connections 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.695+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2053949590 wait complete. 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.695+0000 7f2dd1c5d740 1 -- start start 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.695+0000 7f2dd1c5d740 1 Processor -- start 2024-09-12T03:15:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e1390dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560e13912fc0 con 0x560e139051e0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560e137f4090 con 0x560e13905b00 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e13913140 con 0x560e139103a0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dd1c5d740 10 client.0 _ll_register_callbacks cb 0x560e13848f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59004/0 (socket says 172.21.15.57:59004) 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 -- 172.21.15.57:0/982453229 learned_addr learned my addr 172.21.15.57:0/982453229 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dc7400640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e1390dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 -- 172.21.15.57:0/982453229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 msgr2=0x560e1390dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e1390dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.696+0000 7f2dcce00640 1 -- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 msgr2=0x560e1390d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dcce00640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dcce00640 1 -- 172.21.15.57:0/982453229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1364a620 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dc7400640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dcce00640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009cb0 tx=0x7f2dc000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb69e83b3e97a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2db2000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc002d060 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f2dc0004260 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc0003c30 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2dc001e070 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dd1c5d740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.697+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e1391ebb0 con 0x560e139051e0 2024-09-12T03:15:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0003c30 con 0x560e139051e0 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2dc0019550 con 0x560e139051e0 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 20 client.15789 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13920070 0x560e13922460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 4 client.15789 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e13878830 con 0x560e13920070 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.698+0000 7f2dc7400640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13920070 0x560e13922460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.699+0000 7f2dc7400640 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13920070 0x560e13922460 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56475e7d7629451 server_cookie=68f732c9d6737ea8 in_seq=0 out_seq=0 2024-09-12T03:15:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.699+0000 7f2db2000640 10 client.15789 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2db2000640 1 -- 172.21.15.57:0/982453229 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e13878830 con 0x560e13920070 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2db2000640 10 client.15789 handle_command_reply: tid=0 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 client.15789 shutdown 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 msgr2=0x560e1390d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009cb0 tx=0x7f2dc000f5c0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 shutdown_connections 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13920070 0x560e13922460 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e139103a0 0x560e1390dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e13905b00 0x560e1390d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/982453229 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e1390d280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2db2000640 0 client.15789 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 >> 172.21.15.57:0/982453229 conn(0x560e13901750 msgr2=0x560e139074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 shutdown_connections 2024-09-12T03:15:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/982453229 wait complete. 2024-09-12T03:15:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.700+0000 7f2dd1c5d740 20 client.15789 trim_cache size 0 max 16384 2024-09-12T03:15:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 Processor -- start 2024-09-12T03:15:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 -- start start 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13928090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139103a0 0x560e13900030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81b3cb0 con 0x560e139103a0 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81b1580 con 0x560e139051e0 2024-09-12T03:15:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81af3f0 con 0x560e13905b00 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:41968/0 (socket says 172.21.15.57:41968) 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dcce00640 1 -- 172.21.15.57:0/2852797208 learned_addr learned my addr 172.21.15.57:0/2852797208 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13928090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.716+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139103a0 0x560e13900030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 -- 172.21.15.57:0/2852797208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 msgr2=0x560e13928090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13928090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 -- 172.21.15.57:0/2852797208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139103a0 msgr2=0x560e13900030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139103a0 0x560e13900030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 -- 172.21.15.57:0/2852797208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e13901580 con 0x560e139051e0 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00099a0 tx=0x7f2dc0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55fd1c38609c70a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dbcc00640 1 -- 172.21.15.57:0/2852797208 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0019b90 con 0x560e139051e0 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dbcc00640 1 -- 172.21.15.57:0/2852797208 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0019d30 con 0x560e139051e0 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.717+0000 7f2dbcc00640 1 -- 172.21.15.57:0/2852797208 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000f890 con 0x560e139051e0 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2852797208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 msgr2=0x560e13927b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00099a0 tx=0x7f2dc0014040 comp rx=0 tx=0).stop 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2852797208 shutdown_connections 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139103a0 0x560e13900030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13928090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2852797208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139051e0 0x560e13927b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2852797208 >> 172.21.15.57:0/2852797208 conn(0x560e13901750 msgr2=0x560e1390bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2852797208 shutdown_connections 2024-09-12T03:15:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2852797208 wait complete. 2024-09-12T03:15:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.718+0000 7f2dd1c5d740 1 -- start start 2024-09-12T03:15:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 Processor -- start 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13909bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139103a0 0x560e1390a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81afa40 con 0x560e139051e0 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81b0eb0 con 0x560e139103a0 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81b0110 con 0x560e13905b00 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dd1c5d740 10 client.0 _ll_register_callbacks cb 0x7f2dc001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dcce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:59024/0 (socket says 172.21.15.57:59024) 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dcce00640 1 -- 172.21.15.57:0/2617125371 learned_addr learned my addr 172.21.15.57:0/2617125371 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139103a0 0x560e1390a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.719+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13909bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 -- 172.21.15.57:0/2617125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 msgr2=0x560e13909bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13909bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 -- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139103a0 msgr2=0x560e1390a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139103a0 0x560e1390a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 -- 172.21.15.57:0/2617125371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e138fcd60 con 0x560e139051e0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dcce00640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009690 tx=0x7f2dc0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78c91873d1dd542 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2db2000640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0019430 con 0x560e139051e0 2024-09-12T03:15:24.724 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f2dc00195d0 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc0019d30 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2dc000f500 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.720+0000 7f2dd1c5d740 10 client.15795 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e13918700 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dc0019d30 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2dc0046070 con 0x560e139051e0 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 10 client.15795 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 20 client.15795 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13918b50 0x560e1398cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 4 client.15795 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e13872b80 con 0x560e13918b50 2024-09-12T03:15:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.721+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13918b50 0x560e1398cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.722+0000 7f2dc7400640 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13918b50 0x560e1398cb30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2d4fb01ee0ba1a server_cookie=6ecf242b07642f56 in_seq=0 out_seq=0 2024-09-12T03:15:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.722+0000 7f2db2000640 10 client.15795 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.726+0000 7f2db2000640 1 -- 172.21.15.57:0/2617125371 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e13872b80 con 0x560e13918b50 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.726+0000 7f2db2000640 10 client.15795 handle_command_reply: tid=0 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.726+0000 7f2dd1c5d740 1 client.15795 shutdown 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.726+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 msgr2=0x560e13909670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.726+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009690 tx=0x7f2dc0009500 comp rx=0 tx=0).stop 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 shutdown_connections 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560e13918b50 0x560e1398cb30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560e139103a0 0x560e1390a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e13905b00 0x560e13909bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 --2- 172.21.15.57:0/2617125371 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560e139051e0 0x560e13909670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 >> 172.21.15.57:0/2617125371 conn(0x560e13901750 msgr2=0x560e1391b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2db2000640 0 client.15795 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 shutdown_connections 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 1 -- 172.21.15.57:0/2617125371 wait complete. 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.727+0000 7f2dd1c5d740 20 client.15795 trim_cache size 0 max 16384 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:15:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 msgr2=0x7f2dac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 0x7f2dac0a52e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x560e139025b0 tx=0x7f2db80d35f0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 msgr2=0x7f2dc819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2db800ea60 tx=0x7f2db800ee60 comp rx=0 tx=0).stop 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 shutdown_connections 2024-09-12T03:15:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dc819ef60 0x7f2dc819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f2dac0a2e20 0x7f2dac0a52e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f2dc81090a0 0x7f2dc819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 --2- 172.21.15.57:0/3305718733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8103f50 0x7f2dc819a6a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 >> 172.21.15.57:0/3305718733 conn(0x7f2dc80ff9e0 msgr2=0x7f2dc8101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 shutdown_connections 2024-09-12T03:15:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:24.728+0000 7f2dcee00640 1 -- 172.21.15.57:0/3305718733 wait complete. 2024-09-12T03:15:26.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:25 smithi057 ceph-mon[30361]: pgmap v295: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:26.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:25 smithi204 ceph-mon[35013]: pgmap v295: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:26.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:25 smithi181 ceph-mon[35320]: pgmap v295: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:28.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:27 smithi057 ceph-mon[30361]: pgmap v296: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:28.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:27 smithi204 ceph-mon[35013]: pgmap v296: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:28.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:27 smithi181 ceph-mon[35320]: pgmap v296: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T03:15:30.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:29 smithi057 ceph-mon[30361]: pgmap v297: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:30.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:29 smithi204 ceph-mon[35013]: pgmap v297: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:30.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:29 smithi181 ceph-mon[35320]: pgmap v297: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:31 smithi057 ceph-mon[30361]: pgmap v298: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:31 smithi204 ceph-mon[35013]: pgmap v298: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:32.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:31 smithi181 ceph-mon[35320]: pgmap v298: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.6 KiB/s wr, 6 op/s 2024-09-12T03:15:34.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:33 smithi057 ceph-mon[30361]: pgmap v299: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T03:15:34.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:33 smithi204 ceph-mon[35013]: pgmap v299: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T03:15:34.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:33 smithi181 ceph-mon[35320]: pgmap v299: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-12T03:15:36.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:35 smithi057 ceph-mon[30361]: pgmap v300: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:36.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:35 smithi204 ceph-mon[35013]: pgmap v300: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:36.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:35 smithi181 ceph-mon[35320]: pgmap v300: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 -- 172.21.15.57:0/2331603640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0112960 msgr2=0x7f97e0114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2331603640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0112960 0x7f97e0114d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ca0 tx=0x7f97d802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 -- 172.21.15.57:0/2331603640 shutdown_connections 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2331603640 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0112960 0x7f97e0114d50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2331603640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0077040 0x7f97e0112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2331603640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0076720 0x7f97e0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.855+0000 7f97e6a00640 1 -- 172.21.15.57:0/2331603640 >> 172.21.15.57:0/2331603640 conn(0x7f97e00ff980 msgr2=0x7f97e0101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.856+0000 7f97e6a00640 1 -- 172.21.15.57:0/2331603640 shutdown_connections 2024-09-12T03:15:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.856+0000 7f97e6a00640 1 -- 172.21.15.57:0/2331603640 wait complete. 2024-09-12T03:15:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.856+0000 7f97e6a00640 1 Processor -- start 2024-09-12T03:15:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 -- start start 2024-09-12T03:15:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0077040 0x7f97e01a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0112960 0x7f97e019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0118510 con 0x7f97e0076720 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0118390 con 0x7f97e0077040 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0118690 con 0x7f97e0112960 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:50852/0 (socket says 172.21.15.57:50852) 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97dfe00640 1 -- 172.21.15.57:0/2056051226 learned_addr learned my addr 172.21.15.57:0/2056051226 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97df400640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0077040 0x7f97e01a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.857+0000 7f97e4a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0112960 0x7f97e019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 -- 172.21.15.57:0/2056051226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0112960 msgr2=0x7f97e019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0112960 0x7f97e019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 -- 172.21.15.57:0/2056051226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0077040 msgr2=0x7f97e01a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0077040 0x7f97e01a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 -- 172.21.15.57:0/2056051226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e019ef70 con 0x7f97e0076720 2024-09-12T03:15:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97d000e9f0 tx=0x7f97d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dcc00640 1 -- 172.21.15.57:0/2056051226 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d000cd80 con 0x7f97e0076720 2024-09-12T03:15:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dcc00640 1 -- 172.21.15.57:0/2056051226 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0010910 con 0x7f97e0076720 2024-09-12T03:15:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97dcc00640 1 -- 172.21.15.57:0/2056051226 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0018990 con 0x7f97e0076720 2024-09-12T03:15:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.858+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e019f260 con 0x7f97e0076720 2024-09-12T03:15:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.859+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e01ab480 con 0x7f97e0076720 2024-09-12T03:15:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 Processor -- start 2024-09-12T03:15:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 -- start start 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97dcc00640 1 -- 172.21.15.57:0/2056051226 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f97d0010ac0 con 0x7f97e0076720 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f5e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f6b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5ec4100 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5e6c1b0 con 0x55a4e5f693a0 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.863+0000 7f97e98ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5ec00c0 con 0x55a4e5f5eb00 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97e4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f5e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38870/0 (socket says 172.21.15.57:38870) 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 -- 172.21.15.57:0/2934554885 learned_addr learned my addr 172.21.15.57:0/2934554885 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97df400640 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f6b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 -- 172.21.15.57:0/2934554885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 msgr2=0x55a4e5f6b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f6b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 -- 172.21.15.57:0/2934554885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 msgr2=0x55a4e5f5e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f5e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 -- 172.21.15.57:0/2934554885 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e5bd7210 con 0x55a4e5f5eb00 2024-09-12T03:15:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.864+0000 7f97dfe00640 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ccd50 tx=0x7f97d00d85f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ddf6de957ee36c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97d5200640 1 -- 172.21.15.57:0/2934554885 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00dc640 con 0x55a4e5f5eb00 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97d5200640 1 -- 172.21.15.57:0/2934554885 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d00e1070 con 0x55a4e5f5eb00 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97d5200640 1 -- 172.21.15.57:0/2934554885 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00db9a0 con 0x55a4e5f5eb00 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97dcc00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 0x7f97c40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97dcc00640 1 -- 172.21.15.57:0/2056051226 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f97d00cfb30 con 0x7f97e0076720 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97e4a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 0x7f97c40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97e98ca740 1 -- 172.21.15.57:0/2934554885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 msgr2=0x55a4e5f68e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.865+0000 7f97e98ca740 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ccd50 tx=0x7f97d00d85f0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 -- 172.21.15.57:0/2934554885 shutdown_connections 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f6b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f68e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 --2- 172.21.15.57:0/2934554885 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f5e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 -- 172.21.15.57:0/2934554885 >> 172.21.15.57:0/2934554885 conn(0x55a4e5f5a750 msgr2=0x55a4e5f5ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 -- 172.21.15.57:0/2934554885 shutdown_connections 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 -- 172.21.15.57:0/2934554885 wait complete. 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e4a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 0x7f97c40a52e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97d8004af0 tx=0x7f97d8005d80 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.866+0000 7f97e98ca740 1 -- start start 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 Processor -- start 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5f6bf70 con 0x55a4e5f5eb00 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5e4d090 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a4e5f6c0f0 con 0x55a4e5f693a0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e98ca740 10 client.0 _ll_register_callbacks cb 0x55a4e5ea1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97e4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:51968/0 (socket says 172.21.15.57:51968) 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.867+0000 7f97dfe00640 1 -- 172.21.15.57:0/3258879963 learned_addr learned my addr 172.21.15.57:0/3258879963 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 -- 172.21.15.57:0/3258879963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f693a0 msgr2=0x55a4e5f66df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 -- 172.21.15.57:0/3258879963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 msgr2=0x55a4e5f668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 -- 172.21.15.57:0/3258879963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e5ca3620 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97df400640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97d00d8af0 tx=0x7f97d00dce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344d40d726837c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97c9600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0097f30 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f97d00cd090 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.868+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d00e1070 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97d009b5f0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 10 client.25952 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4e5f77bf0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d00e1070 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f97d009bef0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 10 client.25952 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 20 client.25952 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5f78f40 0x55a4e5f7b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 4 client.25952 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.869+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4e5ed1830 con 0x55a4e5f78f40 2024-09-12T03:15:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.870+0000 7f97df400640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5f78f40 0x55a4e5f7b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.870+0000 7f97df400640 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5f78f40 0x55a4e5f7b330 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee54b8063eac0a56 server_cookie=9a2519778ac3ecb3 in_seq=0 out_seq=0 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.870+0000 7f97c9600640 10 client.25952 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97c9600640 1 -- 172.21.15.57:0/3258879963 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4e5ed1830 con 0x55a4e5f78f40 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97c9600640 10 client.25952 handle_command_reply: tid=0 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 client.25952 shutdown 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 msgr2=0x55a4e5f66370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97d00d8af0 tx=0x7f97d00dce80 comp rx=0 tx=0).stop 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 shutdown_connections 2024-09-12T03:15:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5f78f40 0x55a4e5f7b330 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3258879963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f66370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97c9600640 0 client.25952 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 >> 172.21.15.57:0/3258879963 conn(0x55a4e5f5a750 msgr2=0x55a4e5f604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.871+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 shutdown_connections 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.872+0000 7f97e98ca740 1 -- 172.21.15.57:0/3258879963 wait complete. 2024-09-12T03:15:37.876 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.872+0000 7f97e98ca740 20 client.25952 trim_cache size 0 max 16384 2024-09-12T03:15:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.887+0000 7f97e98ca740 1 Processor -- start 2024-09-12T03:15:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.887+0000 7f97e98ca740 1 -- start start 2024-09-12T03:15:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.887+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.887+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e98ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b80e0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e98ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b59b0 con 0x55a4e5f693a0 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e98ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b3820 con 0x55a4e5f5eb00 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54706/0 (socket says 172.21.15.57:54706) 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97dfe00640 1 -- 172.21.15.57:0/222206859 learned_addr learned my addr 172.21.15.57:0/222206859 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97df400640 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 -- 172.21.15.57:0/222206859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 msgr2=0x55a4e5f74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 -- 172.21.15.57:0/222206859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 msgr2=0x55a4e5f558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97e4a00640 1 -- 172.21.15.57:0/222206859 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e5f6b790 con 0x55a4e5f693a0 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.888+0000 7f97dfe00640 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.889+0000 7f97e4a00640 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97d802ade0 tx=0x7f97d8041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b66140487e8e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.889+0000 7f97cbe00640 1 -- 172.21.15.57:0/222206859 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8003bf0 con 0x55a4e5f693a0 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.889+0000 7f97cbe00640 1 -- 172.21.15.57:0/222206859 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d8036050 con 0x55a4e5f693a0 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.889+0000 7f97cbe00640 1 -- 172.21.15.57:0/222206859 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80044b0 con 0x55a4e5f693a0 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- 172.21.15.57:0/222206859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 msgr2=0x55a4e5f66c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66c20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f97d802ade0 tx=0x7f97d8041000 comp rx=0 tx=0).stop 2024-09-12T03:15:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- 172.21.15.57:0/222206859 shutdown_connections 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f693a0 0x55a4e5f66c20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 --2- 172.21.15.57:0/222206859 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- 172.21.15.57:0/222206859 >> 172.21.15.57:0/222206859 conn(0x55a4e5f5a750 msgr2=0x55a4e5f74340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- 172.21.15.57:0/222206859 shutdown_connections 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- 172.21.15.57:0/222206859 wait complete. 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 -- start start 2024-09-12T03:15:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.890+0000 7f97e98ca740 1 Processor -- start 2024-09-12T03:15:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 0x55a4e5fda830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b3e70 con 0x55a4e5f693a0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b52e0 con 0x55a4e5f5eb00 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b4540 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e98ca740 10 client.0 _ll_register_callbacks cb 0x7f97cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e4a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 0x55a4e5fda830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:38906/0 (socket says 172.21.15.57:38906) 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 -- 172.21.15.57:0/3768853966 learned_addr learned my addr 172.21.15.57:0/3768853966 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97df400640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 -- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5eb00 msgr2=0x55a4e5f7c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f7c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 -- 172.21.15.57:0/3768853966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 msgr2=0x55a4e5fda830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 0x55a4e5fda830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97dfe00640 1 -- 172.21.15.57:0/3768853966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e5fdaed0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.891+0000 7f97e4a00640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 0x55a4e5fda830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97dfe00640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f97d00980f0 tx=0x7f97d00d35e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afd8eb0ad514682 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97c9600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d009b7d0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f97d00a43c0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d8036050 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97d009ba60 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97e98ca740 10 client.25981 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.892+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4e5fdaed0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d8036050 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f97d001abf0 con 0x55a4e5f5e1e0 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 10 client.25981 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 20 client.25981 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5fe3740 0x55a4e5fe5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 4 client.25981 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4e5ecbb80 con 0x55a4e5fe3740 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.893+0000 7f97df400640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5fe3740 0x55a4e5fe5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.894+0000 7f97df400640 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5fe3740 0x55a4e5fe5b30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5ddf8f868b7af1 server_cookie=61f7418ed8068b6c in_seq=0 out_seq=0 2024-09-12T03:15:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.894+0000 7f97c9600640 10 client.25981 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.898+0000 7f97c9600640 1 -- 172.21.15.57:0/3768853966 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4e5ecbb80 con 0x55a4e5fe3740 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.898+0000 7f97c9600640 10 client.25981 handle_command_reply: tid=0 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.898+0000 7f97e98ca740 1 client.25981 shutdown 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 msgr2=0x55a4e5f7bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f97d00980f0 tx=0x7f97d00d35e0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 shutdown_connections 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55a4e5fe3740 0x55a4e5fe5b30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55a4e5f693a0 0x55a4e5fda830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a4e5f5eb00 0x55a4e5f7c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 --2- 172.21.15.57:0/3768853966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a4e5f5e1e0 0x55a4e5f7bbe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97c9600640 0 client.25981 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 >> 172.21.15.57:0/3768853966 conn(0x55a4e5f5a750 msgr2=0x55a4e5f59830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 shutdown_connections 2024-09-12T03:15:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 1 -- 172.21.15.57:0/3768853966 wait complete. 2024-09-12T03:15:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e98ca740 20 client.25981 trim_cache size 0 max 16384 2024-09-12T03:15:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:15:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:15:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:15:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 msgr2=0x7f97c40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 0x7f97c40a52e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97d8004af0 tx=0x7f97d8005d80 comp rx=0 tx=0).stop 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 msgr2=0x7f97e01a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.899+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97d000e9f0 tx=0x7f97d000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 shutdown_connections 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97e0112960 0x7f97e019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97e0077040 0x7f97e01a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f97c40a2e20 0x7f97c40a52e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 --2- 172.21.15.57:0/2056051226 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f97e0076720 0x7f97e01a4630 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 >> 172.21.15.57:0/2056051226 conn(0x7f97e00ff980 msgr2=0x7f97e0101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 shutdown_connections 2024-09-12T03:15:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:37.900+0000 7f97e6a00640 1 -- 172.21.15.57:0/2056051226 wait complete. 2024-09-12T03:15:38.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:37 smithi204 ceph-mon[35013]: pgmap v301: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:38.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:37 smithi181 ceph-mon[35320]: pgmap v301: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:38.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:37 smithi057 ceph-mon[30361]: pgmap v301: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:40.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:39 smithi204 ceph-mon[35013]: pgmap v302: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:40.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:39 smithi181 ceph-mon[35320]: pgmap v302: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:40.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:39 smithi057 ceph-mon[30361]: pgmap v302: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:42.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:41 smithi204 ceph-mon[35013]: pgmap v303: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:15:42.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:41 smithi181 ceph-mon[35320]: pgmap v303: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:15:42.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:41 smithi057 ceph-mon[30361]: pgmap v303: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-12T03:15:44.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:43 smithi181 ceph-mon[35320]: pgmap v304: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:44.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:43 smithi057 ceph-mon[30361]: pgmap v304: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:44.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:43 smithi204 ceph-mon[35013]: pgmap v304: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:46.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:45 smithi181 ceph-mon[35320]: pgmap v305: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:46.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:45 smithi057 ceph-mon[30361]: pgmap v305: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:46.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:45 smithi204 ceph-mon[35013]: pgmap v305: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:47 smithi181 ceph-mon[35320]: pgmap v306: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:48.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:47 smithi057 ceph-mon[30361]: pgmap v306: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:48.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:47 smithi204 ceph-mon[35013]: pgmap v306: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:50.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:49 smithi181 ceph-mon[35320]: pgmap v307: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:50.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:49 smithi057 ceph-mon[30361]: pgmap v307: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:50.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:49 smithi204 ceph-mon[35013]: pgmap v307: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 -- 172.21.15.57:0/102498197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78103af0 msgr2=0x7f4d78103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 --2- 172.21.15.57:0/102498197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78103af0 0x7f4d78103f50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009930 tx=0x7f4d6002a440 comp rx=0 tx=0).stop 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 -- 172.21.15.57:0/102498197 shutdown_connections 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 --2- 172.21.15.57:0/102498197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78104490 0x7f4d781109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 --2- 172.21.15.57:0/102498197 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78103af0 0x7f4d78103f50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 --2- 172.21.15.57:0/102498197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78109b90 0x7f4d78109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 -- 172.21.15.57:0/102498197 >> 172.21.15.57:0/102498197 conn(0x7f4d780ff790 msgr2=0x7f4d78101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 -- 172.21.15.57:0/102498197 shutdown_connections 2024-09-12T03:15:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.026+0000 7f4d7d200640 1 -- 172.21.15.57:0/102498197 wait complete. 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.027+0000 7f4d7d200640 1 Processor -- start 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.027+0000 7f4d7d200640 1 -- start start 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 0x7f4d7819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 0x7f4d7819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78113f40 con 0x7f4d78109b90 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d78113dc0 con 0x7f4d78104490 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d7d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781140c0 con 0x7f4d78103af0 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 0x7f4d7819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:44566/0 (socket says 172.21.15.57:44566) 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 -- 172.21.15.57:0/1565433518 learned_addr learned my addr 172.21.15.57:0/1565433518 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 0x7f4d7819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 -- 172.21.15.57:0/1565433518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 msgr2=0x7f4d7819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 0x7f4d7819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 -- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 msgr2=0x7f4d7819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.028+0000 7f4d77400640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 0x7f4d7819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d77400640 1 -- 172.21.15.57:0/1565433518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7819f440 con 0x7f4d78103af0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 0x7f4d7819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d77e00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 0x7f4d7819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d77400640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9f0 tx=0x7f4d6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d74c00640 1 -- 172.21.15.57:0/1565433518 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800cd80 con 0x7f4d78103af0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7819f730 con 0x7f4d78103af0 2024-09-12T03:15:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d74c00640 1 -- 172.21.15.57:0/1565433518 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d68010910 con 0x7f4d78103af0 2024-09-12T03:15:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d74c00640 1 -- 172.21.15.57:0/1565433518 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680188f0 con 0x7f4d78103af0 2024-09-12T03:15:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.029+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a6fc0 con 0x7f4d78103af0 2024-09-12T03:15:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.032+0000 7f4d74c00640 1 -- 172.21.15.57:0/1565433518 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f4d68010ac0 con 0x7f4d78103af0 2024-09-12T03:15:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.032+0000 7f4d815f2740 1 Processor -- start 2024-09-12T03:15:51.033 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 -- start start 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 0x5598b95fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95fe570 0x5598b9600960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598b9559100 con 0x5598b95fe570 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5598b95011b0 con 0x5598b95f33b0 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d815f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598b95550c0 con 0x5598b95f3cd0 2024-09-12T03:15:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d77e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:53752/0 (socket says 172.21.15.57:53752) 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d77e00640 1 -- 172.21.15.57:0/2284357729 learned_addr learned my addr 172.21.15.57:0/2284357729 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95fe570 0x5598b9600960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.033+0000 7f4d77400640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 0x5598b95fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 -- 172.21.15.57:0/2284357729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 msgr2=0x5598b95f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 -- 172.21.15.57:0/2284357729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95fe570 msgr2=0x5598b9600960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95fe570 0x5598b9600960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 -- 172.21.15.57:0/2284357729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b926c210 con 0x5598b95f3cd0 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77e00640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d74c00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 0x7f4d5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77400640 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 0x5598b95fe030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ccef0 tx=0x7f4d680d86f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3b8773e987f05b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d74c00640 1 -- 172.21.15.57:0/1565433518 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f4d680cfbb0 con 0x7f4d78103af0 2024-09-12T03:15:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d77e00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 0x7f4d5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.034+0000 7f4d6cc00640 1 -- 172.21.15.57:0/2284357729 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68097f60 con 0x5598b95f3cd0 2024-09-12T03:15:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d6cc00640 1 -- 172.21.15.57:0/2284357729 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6809c3c0 con 0x5598b95f3cd0 2024-09-12T03:15:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d6cc00640 1 -- 172.21.15.57:0/2284357729 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6809b620 con 0x5598b95f3cd0 2024-09-12T03:15:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 -- 172.21.15.57:0/2284357729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 msgr2=0x5598b95fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 0x5598b95fe030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ccef0 tx=0x7f4d680d86f0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d77e00640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 0x7f4d5c0a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009fd0 tx=0x7f4d70009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 -- 172.21.15.57:0/2284357729 shutdown_connections 2024-09-12T03:15:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95fe570 0x5598b9600960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95f3cd0 0x5598b95fe030 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2284357729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 -- 172.21.15.57:0/2284357729 >> 172.21.15.57:0/2284357729 conn(0x5598b95ef920 msgr2=0x5598b95efd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 -- 172.21.15.57:0/2284357729 shutdown_connections 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.035+0000 7f4d815f2740 1 -- 172.21.15.57:0/2284357729 wait complete. 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 -- start start 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 Processor -- start 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b95fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598b9600ff0 con 0x5598b95f33b0 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.036+0000 7f4d815f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5598b94e2090 con 0x5598b95f3cd0 2024-09-12T03:15:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d815f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598b9601170 con 0x5598b95fe570 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d815f2740 10 client.0 _ll_register_callbacks cb 0x5598b9536f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44900/0 (socket says 172.21.15.57:44900) 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 -- 172.21.15.57:0/3211947410 learned_addr learned my addr 172.21.15.57:0/3211947410 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77e00640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b95fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 -- 172.21.15.57:0/3211947410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 msgr2=0x5598b95fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 -- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 msgr2=0x5598b95fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b95fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 -- 172.21.15.57:0/3211947410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b9338620 con 0x5598b95f33b0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d77400640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ccfa0 tx=0x7f4d680d3250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9180982add341dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.037+0000 7f4d65600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6809b800 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f4d68099fc0 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6809c3c0 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d6809ba90 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598b960cda0 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d6809c3c0 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4d680cd170 con 0x5598b95f33b0 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.038+0000 7f4d815f2740 20 client.15825 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b960e0f0 0x5598b96104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d815f2740 4 client.15825 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:15:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b9566830 con 0x5598b960e0f0 2024-09-12T03:15:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b960e0f0 0x5598b96104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b960e0f0 0x5598b96104e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e50492998a76df4 server_cookie=f35104d8caea3d72 in_seq=0 out_seq=0 2024-09-12T03:15:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.039+0000 7f4d65600640 10 client.15825 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d65600640 1 -- 172.21.15.57:0/3211947410 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598b9566830 con 0x5598b960e0f0 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d65600640 10 client.15825 handle_command_reply: tid=0 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 client.15825 shutdown 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 msgr2=0x5598b95fb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4d680ccfa0 tx=0x7f4d680d3250 comp rx=0 tx=0).stop 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 shutdown_connections 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b960e0f0 0x5598b96104e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b95fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d815f2740 1 --2- 172.21.15.57:0/3211947410 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b95fb680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.040+0000 7f4d65600640 0 client.15825 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.041+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 >> 172.21.15.57:0/3211947410 conn(0x5598b95ef920 msgr2=0x5598b95f5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.041+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 shutdown_connections 2024-09-12T03:15:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.041+0000 7f4d815f2740 1 -- 172.21.15.57:0/3211947410 wait complete. 2024-09-12T03:15:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.041+0000 7f4d815f2740 20 client.15825 trim_cache size 0 max 16384 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 Processor -- start 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 -- start start 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781b3af0 con 0x5598b95f3cd0 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781b13c0 con 0x5598b95f33b0 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.056+0000 7f4d815f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781af230 con 0x5598b95fe570 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44912/0 (socket says 172.21.15.57:44912) 2024-09-12T03:15:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 -- 172.21.15.57:0/2877062284 learned_addr learned my addr 172.21.15.57:0/2877062284 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d77400640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d77e00640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 -- 172.21.15.57:0/2877062284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 msgr2=0x5598b95ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 -- 172.21.15.57:0/2877062284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 msgr2=0x5598b95f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 -- 172.21.15.57:0/2877062284 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b95f3b00 con 0x5598b95f3cd0 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d77e00640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d77400640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:15:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.057+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4d60000c00 tx=0x7f4d60005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbc6369bf56ca74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d67e00640 1 -- 172.21.15.57:0/2877062284 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60049070 con 0x5598b95f3cd0 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d67e00640 1 -- 172.21.15.57:0/2877062284 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60005520 con 0x5598b95f3cd0 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d67e00640 1 -- 172.21.15.57:0/2877062284 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60035040 con 0x5598b95f3cd0 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 -- 172.21.15.57:0/2877062284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 msgr2=0x5598b95f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4d60000c00 tx=0x7f4d60005b50 comp rx=0 tx=0).stop 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 -- 172.21.15.57:0/2877062284 shutdown_connections 2024-09-12T03:15:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b95ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f3cd0 0x5598b95f8e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 --2- 172.21.15.57:0/2877062284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f33b0 0x5598b95f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 -- 172.21.15.57:0/2877062284 >> 172.21.15.57:0/2877062284 conn(0x5598b95ef920 msgr2=0x5598b95f4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 -- 172.21.15.57:0/2877062284 shutdown_connections 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.058+0000 7f4d815f2740 1 -- 172.21.15.57:0/2877062284 wait complete. 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 -- start start 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 Processor -- start 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b9611140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b96118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781af880 con 0x5598b95f33b0 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781b0cf0 con 0x5598b95f3cd0 2024-09-12T03:15:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d781aff50 con 0x5598b95fe570 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.059+0000 7f4d815f2740 10 client.0 _ll_register_callbacks cb 0x7f4d68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:44916/0 (socket says 172.21.15.57:44916) 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 -- 172.21.15.57:0/1643554385 learned_addr learned my addr 172.21.15.57:0/1643554385 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77e00640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b96118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b9611140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 -- 172.21.15.57:0/1643554385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 msgr2=0x5598b96118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b96118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 -- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 msgr2=0x5598b9611140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b9611140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 -- 172.21.15.57:0/1643554385 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b9677a30 con 0x5598b95f33b0 2024-09-12T03:15:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d77400640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d68098690 tx=0x7f4d680d3fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3006630329f4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.060+0000 7f4d65600640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6809b800 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f4d6809bff0 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d60005520 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d680ae050 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 10 client.15831 fetch_fsmap learned FSMap version 30 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598b9678b90 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60005520 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4d680a44a0 con 0x5598b95f33b0 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 20 client.15831 populate_metadata read hostname 'smithi057' 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b9679060 0x5598b967b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 4 client.15831 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:15:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.061+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598b9560b80 con 0x5598b9679060 2024-09-12T03:15:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.062+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b9679060 0x5598b967b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.062+0000 7f4d6fe00640 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b9679060 0x5598b967b450 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf8b8cb107911ea server_cookie=11da08a83b750e2c in_seq=0 out_seq=0 2024-09-12T03:15:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.062+0000 7f4d65600640 10 client.15831 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.066+0000 7f4d65600640 1 -- 172.21.15.57:0/1643554385 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598b9560b80 con 0x5598b9679060 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.066+0000 7f4d65600640 10 client.15831 handle_command_reply: tid=0 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.066+0000 7f4d815f2740 1 client.15831 shutdown 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 msgr2=0x5598b9613db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d68098690 tx=0x7f4d680d3fa0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 shutdown_connections 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598b9679060 0x5598b967b450 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598b95fe570 0x5598b96118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598b95f3cd0 0x5598b9611140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 --2- 172.21.15.57:0/1643554385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598b95f33b0 0x5598b9613db0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d65600640 0 client.15831 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 >> 172.21.15.57:0/1643554385 conn(0x5598b95ef920 msgr2=0x5598b96093c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 shutdown_connections 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 1 -- 172.21.15.57:0/1643554385 wait complete. 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.067+0000 7f4d815f2740 20 client.15831 trim_cache size 0 max 16384 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:15:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:15:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 msgr2=0x7f4d5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 0x7f4d5c0a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009fd0 tx=0x7f4d70009500 comp rx=0 tx=0).stop 2024-09-12T03:15:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 msgr2=0x7f4d7819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800e9f0 tx=0x7f4d6800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 shutdown_connections 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f4d5c0a2d70 0x7f4d5c0a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4d78109b90 0x7f4d7819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d78104490 0x7f4d7819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 --2- 172.21.15.57:0/1565433518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d78103af0 0x7f4d7819a450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 >> 172.21.15.57:0/1565433518 conn(0x7f4d780ff790 msgr2=0x7f4d78100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 shutdown_connections 2024-09-12T03:15:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:15:51.068+0000 7f4d7d200640 1 -- 172.21.15.57:0/1565433518 wait complete. 2024-09-12T03:15:52.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:51 smithi204 ceph-mon[35013]: pgmap v308: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:52.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:51 smithi181 ceph-mon[35320]: pgmap v308: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:52.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:51 smithi057 ceph-mon[30361]: pgmap v308: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:54.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:53 smithi181 ceph-mon[35320]: pgmap v309: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:15:54.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:53 smithi057 ceph-mon[30361]: pgmap v309: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:15:54.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:53 smithi204 ceph-mon[35013]: pgmap v309: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-12T03:15:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:55 smithi181 ceph-mon[35320]: pgmap v310: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:56.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:55 smithi057 ceph-mon[30361]: pgmap v310: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:56.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:55 smithi204 ceph-mon[35013]: pgmap v310: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:57 smithi181 ceph-mon[35320]: pgmap v311: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:58.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:57 smithi057 ceph-mon[30361]: pgmap v311: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:15:58.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:57 smithi204 ceph-mon[35013]: pgmap v311: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:15:59 smithi181 ceph-mon[35320]: pgmap v312: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:00.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:15:59 smithi057 ceph-mon[30361]: pgmap v312: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:00.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:15:59 smithi204 ceph-mon[35013]: pgmap v312: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:01 smithi181 ceph-mon[35320]: pgmap v313: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:02.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:01 smithi057 ceph-mon[30361]: pgmap v313: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:02.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:01 smithi204 ceph-mon[35013]: pgmap v313: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 -- 172.21.15.57:0/1854914979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63001090a0 msgr2=0x7f6300110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 --2- 172.21.15.57:0/1854914979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63001090a0 0x7f6300110bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009ca0 tx=0x7f62f802a8b0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 -- 172.21.15.57:0/1854914979 shutdown_connections 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 --2- 172.21.15.57:0/1854914979 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f63001090a0 0x7f6300110bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 --2- 172.21.15.57:0/1854914979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6300104900 0x7f6300108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 --2- 172.21.15.57:0/1854914979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6300103f50 0x7f6300104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 -- 172.21.15.57:0/1854914979 >> 172.21.15.57:0/1854914979 conn(0x7f63000ff9e0 msgr2=0x7f6300101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 -- 172.21.15.57:0/1854914979 shutdown_connections 2024-09-12T03:16:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.197+0000 7f6305600640 1 -- 172.21.15.57:0/1854914979 wait complete. 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 Processor -- start 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 -- start start 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63001090a0 0x7f630019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f630019ede0 0x7f630019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6300114140 con 0x7f6300103f50 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6300113fc0 con 0x7f63001090a0 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f6305600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63001142c0 con 0x7f630019ede0 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.198+0000 7f62fea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39394/0 (socket says 172.21.15.57:39394) 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 -- 172.21.15.57:0/85963239 learned_addr learned my addr 172.21.15.57:0/85963239 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fe000640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63001090a0 0x7f630019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62ff400640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f630019ede0 0x7f630019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 -- 172.21.15.57:0/85963239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f630019ede0 msgr2=0x7f630019f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f630019ede0 0x7f630019f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 -- 172.21.15.57:0/85963239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63001090a0 msgr2=0x7f630019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63001090a0 0x7f630019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 -- 172.21.15.57:0/85963239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019f8d0 con 0x7f6300103f50 2024-09-12T03:16:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.199+0000 7f62fea00640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f62f000e9f0 tx=0x7f62f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.200+0000 7f62f7400640 1 -- 172.21.15.57:0/85963239 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f000cd80 con 0x7f6300103f50 2024-09-12T03:16:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.200+0000 7f62f7400640 1 -- 172.21.15.57:0/85963239 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f0005500 con 0x7f6300103f50 2024-09-12T03:16:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.200+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019fbc0 con 0x7f6300103f50 2024-09-12T03:16:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.200+0000 7f62f7400640 1 -- 172.21.15.57:0/85963239 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0010690 con 0x7f6300103f50 2024-09-12T03:16:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.200+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a7360 con 0x7f6300103f50 2024-09-12T03:16:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f62f7400640 1 -- 172.21.15.57:0/85963239 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f62f00046c0 con 0x7f6300103f50 2024-09-12T03:16:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 Processor -- start 2024-09-12T03:16:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 -- start start 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 0x55c12ea7fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea82790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c12e9db100 con 0x55c12ea751e0 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c12e9831b0 con 0x55c12ea75b00 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.203+0000 7f630848e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c12e9d70c0 con 0x55c12ea803a0 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62fea00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 0x55c12ea7fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39398/0 (socket says 172.21.15.57:39398) 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 -- 172.21.15.57:0/1686431521 learned_addr learned my addr 172.21.15.57:0/1686431521 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62fe000640 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea82790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 -- 172.21.15.57:0/1686431521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 msgr2=0x55c12ea82790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea82790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 -- 172.21.15.57:0/1686431521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 msgr2=0x55c12ea7fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 0x55c12ea7fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62ff400640 1 -- 172.21.15.57:0/1686431521 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12e6ee210 con 0x55c12ea751e0 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62fea00640 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 0x55c12ea7fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62f7400640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 0x7f62e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.204+0000 7f62f7400640 1 -- 172.21.15.57:0/85963239 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f62f00cfca0 con 0x7f6300103f50 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.205+0000 7f62ff400640 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009ca0 tx=0x7f62f80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e39efc8627450b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.205+0000 7f62fea00640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 0x7f62e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.205+0000 7f62eb400640 1 -- 172.21.15.57:0/1686431521 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8030760 con 0x55c12ea751e0 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.205+0000 7f62eb400640 1 -- 172.21.15.57:0/1686431521 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f8036030 con 0x55c12ea751e0 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.205+0000 7f62eb400640 1 -- 172.21.15.57:0/1686431521 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8004360 con 0x55c12ea751e0 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- 172.21.15.57:0/1686431521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 msgr2=0x55c12ea755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f62fea00640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 0x7f62e40a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c12ea725b0 tx=0x7f62f00d3750 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009ca0 tx=0x7f62f80093e0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- 172.21.15.57:0/1686431521 shutdown_connections 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea82790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea75b00 0x55c12ea7fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 --2- 172.21.15.57:0/1686431521 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea755c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- 172.21.15.57:0/1686431521 >> 172.21.15.57:0/1686431521 conn(0x55c12ea71750 msgr2=0x55c12ea71b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- 172.21.15.57:0/1686431521 shutdown_connections 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- 172.21.15.57:0/1686431521 wait complete. 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.206+0000 7f630848e740 1 -- start start 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 Processor -- start 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea803a0 0x55c12ea7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55c12ea82f70 con 0x55c12ea751e0 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55c12e964090 con 0x55c12ea803a0 2024-09-12T03:16:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c12ea830f0 con 0x55c12ea75b00 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f630848e740 10 client.0 _ll_register_callbacks cb 0x55c12e9b8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f62ff400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f62ff400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:39406/0 (socket says 172.21.15.57:39406) 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f62ff400640 1 -- 172.21.15.57:0/3212265510 learned_addr learned my addr 172.21.15.57:0/3212265510 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.207+0000 7f62fe000640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62ff400640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea803a0 0x55c12ea7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 -- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea803a0 msgr2=0x55c12ea7dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea803a0 0x55c12ea7dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 -- 172.21.15.57:0/3212265510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 msgr2=0x55c12ea7d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 -- 172.21.15.57:0/3212265510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12e7ba620 con 0x55c12ea75b00 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62ff400640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62fe000640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea7d810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55c12ea77d80 tx=0x7f62ec012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=419d81a590e0743e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.208+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec004030 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f8036030 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f62ec0041d0 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62ec0043d0 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 10 client.26011 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c12ea8ec00 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f8036030 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f62ec003ac0 con 0x55c12ea75b00 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 10 client.26011 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 20 client.26011 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12ea90070 0x55c12ea92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 4 client.26011 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.209+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c12e9e8830 con 0x55c12ea90070 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.210+0000 7f62ff400640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12ea90070 0x55c12ea92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.210+0000 7f62ff400640 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12ea90070 0x55c12ea92460 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad06c4ee7ad1f96 server_cookie=3def0791b7f8a908 in_seq=0 out_seq=0 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.210+0000 7f62dfe00640 10 client.26011 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.211+0000 7f62dfe00640 1 -- 172.21.15.57:0/3212265510 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c12e9e8830 con 0x55c12ea90070 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.211+0000 7f62dfe00640 10 client.26011 handle_command_reply: tid=0 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.211+0000 7f630848e740 1 client.26011 shutdown 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.211+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 msgr2=0x55c12ea7d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.211+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea7d810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x55c12ea77d80 tx=0x7f62ec012b60 comp rx=0 tx=0).stop 2024-09-12T03:16:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 shutdown_connections 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12ea90070 0x55c12ea92460 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea803a0 0x55c12ea7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea7d810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f62dfe00640 0 client.26011 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 --2- 172.21.15.57:0/3212265510 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea751e0 0x55c12ea7d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 >> 172.21.15.57:0/3212265510 conn(0x55c12ea71750 msgr2=0x55c12ea774b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 shutdown_connections 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 1 -- 172.21.15.57:0/3212265510 wait complete. 2024-09-12T03:16:04.216 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.212+0000 7f630848e740 20 client.26011 trim_cache size 0 max 16384 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.227+0000 7f630848e740 1 Processor -- start 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.227+0000 7f630848e740 1 -- start start 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea803a0 0x55c12ea76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f63001b3b00 con 0x55c12ea803a0 2024-09-12T03:16:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63001b13d0 con 0x55c12ea751e0 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f630848e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63001af240 con 0x55c12ea75b00 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea803a0 0x55c12ea76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62fe000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:60916/0 (socket says 172.21.15.57:60916) 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62fe000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:49406/0 (socket says 172.21.15.57:49406) 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 -- 172.21.15.57:0/2726059698 learned_addr learned my addr 172.21.15.57:0/2726059698 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 -- 172.21.15.57:0/2726059698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 msgr2=0x55c12ea982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 -- 172.21.15.57:0/2726059698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea803a0 msgr2=0x55c12ea76c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea803a0 0x55c12ea76c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62ff400640 1 -- 172.21.15.57:0/2726059698 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12ea75930 con 0x55c12ea751e0 2024-09-12T03:16:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.228+0000 7f62fe000640 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.229+0000 7f62ff400640 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f62f8030fd0 tx=0x7f62f8031000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02ca2f3044cd95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.229+0000 7f62eaa00640 1 -- 172.21.15.57:0/2726059698 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8030c50 con 0x55c12ea751e0 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.229+0000 7f62eaa00640 1 -- 172.21.15.57:0/2726059698 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f8036040 con 0x55c12ea751e0 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.229+0000 7f62eaa00640 1 -- 172.21.15.57:0/2726059698 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8045440 con 0x55c12ea751e0 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- 172.21.15.57:0/2726059698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 msgr2=0x55c12ea97d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f62f8030fd0 tx=0x7f62f8031000 comp rx=0 tx=0).stop 2024-09-12T03:16:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- 172.21.15.57:0/2726059698 shutdown_connections 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea803a0 0x55c12ea76c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea75b00 0x55c12ea982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 --2- 172.21.15.57:0/2726059698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea97d70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- 172.21.15.57:0/2726059698 >> 172.21.15.57:0/2726059698 conn(0x55c12ea71750 msgr2=0x55c12ea7bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- 172.21.15.57:0/2726059698 shutdown_connections 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- 172.21.15.57:0/2726059698 wait complete. 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 -- start start 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.230+0000 7f630848e740 1 Processor -- start 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea88500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 0x55c12ea88a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f63001af890 con 0x55c12ea75b00 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f63001b0d00 con 0x55c12ea751e0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63001aff60 con 0x55c12ea803a0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f630848e740 10 client.0 _ll_register_callbacks cb 0x7f62ec009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f62ff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea798f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f62fe000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 0x55c12ea88a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f62ff400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea798f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:49422/0 (socket says 172.21.15.57:49422) 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f62ff400640 1 -- 172.21.15.57:0/350832290 learned_addr learned my addr 172.21.15.57:0/350832290 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.231+0000 7f62ff400640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea88500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 -- 172.21.15.57:0/350832290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 msgr2=0x55c12ea798f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea798f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 -- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 msgr2=0x55c12ea88500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea88500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 -- 172.21.15.57:0/350832290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12ea79f30 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62fe000640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 0x55c12ea88a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01c500 tx=0x7f62ec009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a97abaeccb3911c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec009e90 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7f62ec01c8b0 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62f8036040 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62ec0041f0 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f630848e740 10 client.15849 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.232+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c12eafa610 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f8036040 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f62ec003710 con 0x55c12ea75b00 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 10 client.15849 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 20 client.15849 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12eafade0 0x55c12eafd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 4 client.15849 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c12e9e2b80 con 0x55c12eafade0 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.233+0000 7f62ff400640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12eafade0 0x55c12eafd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.234+0000 7f62ff400640 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12eafade0 0x55c12eafd200 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c6badb6912b913 server_cookie=119554a7ff7cc440 in_seq=0 out_seq=0 2024-09-12T03:16:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.234+0000 7f62dfe00640 10 client.15849 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f62dfe00640 1 -- 172.21.15.57:0/350832290 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c12e9e2b80 con 0x55c12eafade0 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f62dfe00640 10 client.15849 handle_command_reply: tid=0 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 client.15849 shutdown 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 msgr2=0x55c12ea88a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 0x55c12ea88a40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f62ec01c500 tx=0x7f62ec009ca0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 shutdown_connections 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55c12eafade0 0x55c12eafd200 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c12ea803a0 0x55c12ea798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55c12ea75b00 0x55c12ea88a40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f62dfe00640 0 client.15849 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 --2- 172.21.15.57:0/350832290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c12ea751e0 0x55c12ea88500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 >> 172.21.15.57:0/350832290 conn(0x55c12ea71750 msgr2=0x55c12ea76c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 shutdown_connections 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 1 -- 172.21.15.57:0/350832290 wait complete. 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.241+0000 7f630848e740 20 client.15849 trim_cache size 0 max 16384 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:16:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 msgr2=0x7f62e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 0x7f62e40a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55c12ea725b0 tx=0x7f62f00d3750 comp rx=0 tx=0).stop 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 msgr2=0x7f630019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f62f000e9f0 tx=0x7f62f000edf0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 shutdown_connections 2024-09-12T03:16:04.244 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f630019ede0 0x7f630019f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63001090a0 0x7f630019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62e40a2da0 0x7f62e40a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 --2- 172.21.15.57:0/85963239 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6300103f50 0x7f630019a520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 >> 172.21.15.57:0/85963239 conn(0x7f63000ff9e0 msgr2=0x7f6300100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 shutdown_connections 2024-09-12T03:16:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:04.242+0000 7f6305600640 1 -- 172.21.15.57:0/85963239 wait complete. 2024-09-12T03:16:04.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:03 smithi057 ceph-mon[30361]: pgmap v314: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:16:04.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:03 smithi204 ceph-mon[35013]: pgmap v314: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:16:04.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:03 smithi181 ceph-mon[35320]: pgmap v314: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 7.1 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:16:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:05 smithi057 ceph-mon[30361]: pgmap v315: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-12T03:16:06.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:05 smithi204 ceph-mon[35013]: pgmap v315: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-12T03:16:06.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:05 smithi181 ceph-mon[35320]: pgmap v315: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 5.8 KiB/s rd, 23 KiB/s wr, 7 op/s 2024-09-12T03:16:07.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:06 smithi057 ceph-mon[30361]: pgmap v316: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-12T03:16:07.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:06 smithi204 ceph-mon[35013]: pgmap v316: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-12T03:16:07.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:06 smithi181 ceph-mon[35320]: pgmap v316: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-12T03:16:08.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:07 smithi057 ceph-mon[30361]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:16:08.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:07 smithi204 ceph-mon[35013]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:16:08.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:07 smithi181 ceph-mon[35320]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T03:16:09.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:08 smithi057 ceph-mon[30361]: pgmap v318: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:09.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:08 smithi204 ceph-mon[35013]: pgmap v318: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:09.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:08 smithi181 ceph-mon[35320]: pgmap v318: 129 pgs: 129 active+clean; 27 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:11.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:11 smithi181 ceph-mon[35320]: pgmap v319: 129 pgs: 129 active+clean; 26 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:11.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:11 smithi057 ceph-mon[30361]: pgmap v319: 129 pgs: 129 active+clean; 26 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:11.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:11 smithi204 ceph-mon[35013]: pgmap v319: 129 pgs: 129 active+clean; 26 GiB data, 57 GiB used, 1016 GiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-09-12T03:16:13.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:13 smithi181 ceph-mon[35320]: pgmap v320: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:13.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:13 smithi057 ceph-mon[30361]: pgmap v320: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:13.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:13 smithi204 ceph-mon[35013]: pgmap v320: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:15.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:15 smithi181 ceph-mon[35320]: pgmap v321: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:15.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:15 smithi057 ceph-mon[30361]: pgmap v321: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:15.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:15 smithi204 ceph-mon[35013]: pgmap v321: 129 pgs: 129 active+clean; 18 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9d5400640 1 -- 172.21.15.57:0/1077781832 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00319b0 con 0x7fb9d81048e0 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 -- 172.21.15.57:0/1077781832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d81048e0 msgr2=0x7fb9d8108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 --2- 172.21.15.57:0/1077781832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d81048e0 0x7fb9d8108920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009900 tx=0x7fb9c002a410 comp rx=0 tx=0).stop 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 -- 172.21.15.57:0/1077781832 shutdown_connections 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 --2- 172.21.15.57:0/1077781832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 0x7fb9d8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 --2- 172.21.15.57:0/1077781832 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d81048e0 0x7fb9d8108920 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 --2- 172.21.15.57:0/1077781832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d8103f30 0x7fb9d8104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 -- 172.21.15.57:0/1077781832 >> 172.21.15.57:0/1077781832 conn(0x7fb9d80ff980 msgr2=0x7fb9d8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 -- 172.21.15.57:0/1077781832 shutdown_connections 2024-09-12T03:16:17.424 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.423+0000 7fb9dce00640 1 -- 172.21.15.57:0/1077781832 wait complete. 2024-09-12T03:16:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.424+0000 7fb9dce00640 1 Processor -- start 2024-09-12T03:16:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.424+0000 7fb9dce00640 1 -- start start 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 0x7fb9d819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d819ee20 0x7fb9d819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81140d0 con 0x7fb9d8103f30 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d8113f50 con 0x7fb9d819ee20 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9dce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d8114250 con 0x7fb9d8109080 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42652/0 (socket says 172.21.15.57:42652) 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 -- 172.21.15.57:0/3439074774 learned_addr learned my addr 172.21.15.57:0/3439074774 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d819ee20 0x7fb9d819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 0x7fb9d819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 -- 172.21.15.57:0/3439074774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 msgr2=0x7fb9d819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 0x7fb9d819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 -- 172.21.15.57:0/3439074774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d819ee20 msgr2=0x7fb9d819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d819ee20 0x7fb9d819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.425+0000 7fb9d5e00640 1 -- 172.21.15.57:0/3439074774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d819f910 con 0x7fb9d8103f30 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e9f0 tx=0x7fb9c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9cea00640 1 -- 172.21.15.57:0/3439074774 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800cd80 con 0x7fb9d8103f30 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9cea00640 1 -- 172.21.15.57:0/3439074774 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8010910 con 0x7fb9d8103f30 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d819fc00 con 0x7fb9d8103f30 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9cea00640 1 -- 172.21.15.57:0/3439074774 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8018990 con 0x7fb9d8103f30 2024-09-12T03:16:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.426+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81a7410 con 0x7fb9d8103f30 2024-09-12T03:16:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.429+0000 7fb9cea00640 1 -- 172.21.15.57:0/3439074774 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fb9c8010ac0 con 0x7fb9d8103f30 2024-09-12T03:16:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 Processor -- start 2024-09-12T03:16:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 -- start start 2024-09-12T03:16:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c1d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c2a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55df49b83100 con 0x55df49c1d1e0 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df49b2b1b0 con 0x55df49c283a0 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.430+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55df49b7f0c0 con 0x55df49c1db00 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9cea00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 0x7fb9bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9cea00640 1 -- 172.21.15.57:0/3439074774 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fb9c80cfb30 con 0x7fb9d8103f30 2024-09-12T03:16:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:34602/0 (socket says 172.21.15.57:34602) 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 -- 172.21.15.57:0/483920837 learned_addr learned my addr 172.21.15.57:0/483920837 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c1d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c2a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d6800640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 0x7fb9bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 -- 172.21.15.57:0/483920837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 msgr2=0x55df49c2a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c2a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 -- 172.21.15.57:0/483920837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 msgr2=0x55df49c1d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c1d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d5e00640 1 -- 172.21.15.57:0/483920837 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df49896210 con 0x55df49c1db00 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.431+0000 7fb9d6800640 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c1d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.432+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e9c0 tx=0x7fb9c809bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f553eac0dce4c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.432+0000 7fb9c7400640 1 -- 172.21.15.57:0/483920837 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8099f90 con 0x55df49c1db00 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.432+0000 7fb9c7400640 1 -- 172.21.15.57:0/483920837 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8098300 con 0x55df49c1db00 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.432+0000 7fb9c7400640 1 -- 172.21.15.57:0/483920837 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c809b2e0 con 0x55df49c1db00 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- 172.21.15.57:0/483920837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 msgr2=0x55df49c27e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e9c0 tx=0x7fb9c809bd50 comp rx=0 tx=0).stop 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9d6800640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 0x7fb9bc0a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009500 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- 172.21.15.57:0/483920837 shutdown_connections 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c2a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1db00 0x55df49c27e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/483920837 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c1d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- 172.21.15.57:0/483920837 >> 172.21.15.57:0/483920837 conn(0x55df49c19750 msgr2=0x55df49c19b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- 172.21.15.57:0/483920837 shutdown_connections 2024-09-12T03:16:17.437 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- 172.21.15.57:0/483920837 wait complete. 2024-09-12T03:16:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.433+0000 7fb9dfca2740 1 -- start start 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 Processor -- start 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55df49c2af70 con 0x55df49c1d1e0 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55df49b0c090 con 0x55df49c1db00 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55df49c2b0f0 con 0x55df49c283a0 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9dfca2740 10 client.0 _ll_register_callbacks cb 0x55df49b60f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.434+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:36270/0 (socket says 172.21.15.57:36270) 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 -- 172.21.15.57:0/3059398039 learned_addr learned my addr 172.21.15.57:0/3059398039 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42668/0 (socket says 172.21.15.57:42668) 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9d6800640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 -- 172.21.15.57:0/3059398039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 msgr2=0x55df49c25e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c25e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 -- 172.21.15.57:0/3059398039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 msgr2=0x55df49c253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 -- 172.21.15.57:0/3059398039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df49962620 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009230 tx=0x7fb9c00317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9445c4ac91c301 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.435+0000 7fb9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0035040 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c8098300 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fb9c0004240 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0004440 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9dfca2740 10 client.26006 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df49c36c00 con 0x55df49c1db00 2024-09-12T03:16:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.436+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c8098300 con 0x55df49c1db00 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c0004bf0 con 0x55df49c1db00 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 10 client.26006 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 10 client.26006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 20 client.26006 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49c380c0 0x55df49c3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 4 client.26006 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df49b90830 con 0x55df49c380c0 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.437+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49c380c0 0x55df49c3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.438+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49c380c0 0x55df49c3a4b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c19c60eafe3079 server_cookie=fe32b388ec8ad77b in_seq=0 out_seq=0 2024-09-12T03:16:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.438+0000 7fb9bbe00640 10 client.26006 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.438+0000 7fb9bbe00640 1 -- 172.21.15.57:0/3059398039 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df49b90830 con 0x55df49c380c0 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.438+0000 7fb9bbe00640 10 client.26006 handle_command_reply: tid=0 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 client.26006 shutdown 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 msgr2=0x55df49c25910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009230 tx=0x7fb9c00317e0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 shutdown_connections 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49c380c0 0x55df49c3a4b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c25e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c25910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9bbe00640 0 client.26006 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/3059398039 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 >> 172.21.15.57:0/3059398039 conn(0x55df49c19750 msgr2=0x55df49c1f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 shutdown_connections 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 1 -- 172.21.15.57:0/3059398039 wait complete. 2024-09-12T03:16:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.439+0000 7fb9dfca2740 20 client.26006 trim_cache size 0 max 16384 2024-09-12T03:16:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 Processor -- start 2024-09-12T03:16:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 -- start start 2024-09-12T03:16:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c402e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c18010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b3b90 con 0x55df49c1d1e0 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b1460 con 0x55df49c1db00 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.455+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81af2d0 con 0x55df49c283a0 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42684/0 (socket says 172.21.15.57:42684) 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 -- 172.21.15.57:0/865378617 learned_addr learned my addr 172.21.15.57:0/865378617 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d6800640 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c18010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 -- 172.21.15.57:0/865378617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 msgr2=0x55df49c18010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c18010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 -- 172.21.15.57:0/865378617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 msgr2=0x55df49c402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 -- 172.21.15.57:0/865378617 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df49c1d010 con 0x55df49c1d1e0 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80980f0 tx=0x7fb9c80a49d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466f28a6bdf04279 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.456+0000 7fb9c6a00640 1 -- 172.21.15.57:0/865378617 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8099f90 con 0x55df49c1d1e0 2024-09-12T03:16:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9c6a00640 1 -- 172.21.15.57:0/865378617 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c80cd090 con 0x55df49c1d1e0 2024-09-12T03:16:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9c6a00640 1 -- 172.21.15.57:0/865378617 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8097c40 con 0x55df49c1d1e0 2024-09-12T03:16:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 -- 172.21.15.57:0/865378617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 msgr2=0x55df49c3fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80980f0 tx=0x7fb9c80a49d0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 -- 172.21.15.57:0/865378617 shutdown_connections 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c283a0 0x55df49c18010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c1db00 0x55df49c402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/865378617 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1d1e0 0x55df49c3fda0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 -- 172.21.15.57:0/865378617 >> 172.21.15.57:0/865378617 conn(0x55df49c19750 msgr2=0x55df49c23c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 -- 172.21.15.57:0/865378617 shutdown_connections 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.457+0000 7fb9dfca2740 1 -- 172.21.15.57:0/865378617 wait complete. 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 -- start start 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 Processor -- start 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 0x55df49c30960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81af920 con 0x55df49c1db00 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81b0d90 con 0x55df49c283a0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.458+0000 7fb9dfca2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81afff0 con 0x55df49c1d1e0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9dfca2740 10 client.0 _ll_register_callbacks cb 0x7fb9c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 0x55df49c30960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:42686/0 (socket says 172.21.15.57:42686) 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9cfe00640 1 -- 172.21.15.57:0/2210016621 learned_addr learned my addr 172.21.15.57:0/2210016621 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d6800640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 -- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 msgr2=0x55df49c225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c225c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 -- 172.21.15.57:0/2210016621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 msgr2=0x55df49c22080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 -- 172.21.15.57:0/2210016621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df49c22da0 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d6800640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9d5e00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 0x55df49c30960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8099f40 tx=0x7fb9c809bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d6571b2c8bb791 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.459+0000 7fb9bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80a8020 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fb9c809b0e0 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c80cd090 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9c8098300 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9dfca2740 10 client.26035 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df49ca2d00 con 0x55df49c1d1e0 2024-09-12T03:16:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c80cd090 con 0x55df49c1d1e0 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.460+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c8097b80 con 0x55df49c1d1e0 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 10 client.26035 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 20 client.26035 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49ca2f80 0x55df49ca5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 4 client.26035 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df49b8ab80 con 0x55df49ca2f80 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49ca2f80 0x55df49ca5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9cfe00640 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49ca2f80 0x55df49ca5370 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4a9870c3935a5e server_cookie=d48e3c2fb9ed6065 in_seq=0 out_seq=0 2024-09-12T03:16:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.461+0000 7fb9bbe00640 10 client.26035 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.468+0000 7fb9bbe00640 1 -- 172.21.15.57:0/2210016621 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df49b8ab80 con 0x55df49ca2f80 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.468+0000 7fb9bbe00640 10 client.26035 handle_command_reply: tid=0 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.468+0000 7fb9dfca2740 1 client.26035 shutdown 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.468+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 msgr2=0x55df49c30960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.468+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 0x55df49c30960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8099f40 tx=0x7fb9c809bd50 comp rx=0 tx=0).stop 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 shutdown_connections 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55df49ca2f80 0x55df49ca5370 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55df49c283a0 0x55df49c225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55df49c1db00 0x55df49c22080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9bbe00640 0 client.26035 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 --2- 172.21.15.57:0/2210016621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55df49c1d1e0 0x55df49c30960 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 >> 172.21.15.57:0/2210016621 conn(0x55df49c19750 msgr2=0x55df49c25310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 shutdown_connections 2024-09-12T03:16:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 1 -- 172.21.15.57:0/2210016621 wait complete. 2024-09-12T03:16:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.469+0000 7fb9dfca2740 20 client.26035 trim_cache size 0 max 16384 2024-09-12T03:16:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:16:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:16:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:16:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 msgr2=0x7fb9bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 0x7fb9bc0a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d0009500 comp rx=0 tx=0).stop 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 msgr2=0x7fb9d819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800e9f0 tx=0x7fb9c800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 shutdown_connections 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9d819ee20 0x7fb9d819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9d8109080 0x7fb9d819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fb9bc0a2e20 0x7fb9bc0a52e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 --2- 172.21.15.57:0/3439074774 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fb9d8103f30 0x7fb9d819a560 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 >> 172.21.15.57:0/3439074774 conn(0x7fb9d80ff980 msgr2=0x7fb9d8101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 shutdown_connections 2024-09-12T03:16:17.476 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:17.470+0000 7fb9dce00640 1 -- 172.21.15.57:0/3439074774 wait complete. 2024-09-12T03:16:17.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:17 smithi181 ceph-mon[35320]: pgmap v322: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:16:17.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:17 smithi181 ceph-mon[35320]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:16:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:17 smithi057 ceph-mon[30361]: pgmap v322: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:16:18.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:17 smithi057 ceph-mon[30361]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:16:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:17 smithi204 ceph-mon[35013]: pgmap v322: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-09-12T03:16:18.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:17 smithi204 ceph-mon[35013]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T03:16:19.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:19 smithi181 ceph-mon[35320]: pgmap v324: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:20.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:19 smithi057 ceph-mon[30361]: pgmap v324: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:20.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:19 smithi204 ceph-mon[35013]: pgmap v324: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:20.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:20 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:20.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:20 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:20 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:20 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.894 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:20 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:20 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:20 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:20 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:20 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:20.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:20 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:20 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:20 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:20 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:20 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:20 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:21 smithi181 ceph-mon[35320]: pgmap v325: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:21 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:21 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:21 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:21.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:21 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:21 smithi057 ceph-mon[30361]: pgmap v325: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:21 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:21 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:21 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:21 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:21 smithi204 ceph-mon[35013]: pgmap v325: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-12T03:16:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:21 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:21 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:21 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:21 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:22.954 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:22 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:16:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:22 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-12T03:16:23.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi204 to 184562380: error parsing value: Value '184562380' is below minimum 939524096 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:23.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:22 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: pgmap v326: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:24.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: pgmap v326: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:24.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi181 to 176.0M 2024-09-12T03:16:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi181 to 184558080: error parsing value: Value '184558080' is below minimum 939524096 2024-09-12T03:16:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: pgmap v326: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:24.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:16:25.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:24 smithi057 ceph-mon[30361]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:16:25.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:24 smithi057 ceph-mon[30361]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:16:25.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:24 smithi204 ceph-mon[35013]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:16:25.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:24 smithi204 ceph-mon[35013]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:16:25.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:24 smithi181 ceph-mon[35320]: Adjusting osd_memory_target on smithi057 to 177.5M 2024-09-12T03:16:25.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:24 smithi181 ceph-mon[35320]: Unable to set osd_memory_target on smithi057 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-12T03:16:26.021 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:25 smithi057 ceph-mon[30361]: pgmap v327: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:26.104 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:25 smithi181 ceph-mon[35320]: pgmap v327: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:26.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:25 smithi204 ceph-mon[35013]: pgmap v327: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:28.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:27 smithi057 ceph-mon[30361]: pgmap v328: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:28.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:27 smithi204 ceph-mon[35013]: pgmap v328: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:28.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:27 smithi181 ceph-mon[35320]: pgmap v328: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.8 KiB/s wr, 6 op/s 2024-09-12T03:16:30.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:29 smithi057 ceph-mon[30361]: pgmap v329: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:16:30.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:29 smithi204 ceph-mon[35013]: pgmap v329: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:16:30.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:29 smithi181 ceph-mon[35320]: pgmap v329: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-12T03:16:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 -- 172.21.15.57:0/1635621087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728103400 msgr2=0x7fa72810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 --2- 172.21.15.57:0/1635621087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728103400 0x7fa72810b8e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa71402a480 comp rx=0 tx=0).stop 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 -- 172.21.15.57:0/1635621087 shutdown_connections 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 --2- 172.21.15.57:0/1635621087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 --2- 172.21.15.57:0/1635621087 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728103400 0x7fa72810b8e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 --2- 172.21.15.57:0/1635621087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728102ae0 0x7fa728102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 -- 172.21.15.57:0/1635621087 >> 172.21.15.57:0/1635621087 conn(0x7fa7280777c0 msgr2=0x7fa7280feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.597+0000 7fa72ec00640 1 -- 172.21.15.57:0/1635621087 shutdown_connections 2024-09-12T03:16:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.598+0000 7fa72ec00640 1 -- 172.21.15.57:0/1635621087 wait complete. 2024-09-12T03:16:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.598+0000 7fa72ec00640 1 Processor -- start 2024-09-12T03:16:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 -- start start 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728103400 0x7fa72819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa728111d20 con 0x7fa728102ae0 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa728111ba0 con 0x7fa72810bf00 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa728111ea0 con 0x7fa728103400 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54876/0 (socket says 172.21.15.57:54876) 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72d800640 1 -- 172.21.15.57:0/353925733 learned_addr learned my addr 172.21.15.57:0/353925733 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72e200640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.599+0000 7fa72ce00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728103400 0x7fa72819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 -- 172.21.15.57:0/353925733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728103400 msgr2=0x7fa72819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728103400 0x7fa72819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 -- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 msgr2=0x7fa72819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 -- 172.21.15.57:0/353925733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72819f420 con 0x7fa72810bf00 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72d800640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72e200640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72819ecf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa720009fd0 tx=0x7fa72000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa726000640 1 -- 172.21.15.57:0/353925733 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72000ce20 con 0x7fa72810bf00 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72819f710 con 0x7fa72810bf00 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa726000640 1 -- 172.21.15.57:0/353925733 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa720005020 con 0x7fa72810bf00 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.600+0000 7fa726000640 1 -- 172.21.15.57:0/353925733 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720010720 con 0x7fa72810bf00 2024-09-12T03:16:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.601+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7281a6fa0 con 0x7fa72810bf00 2024-09-12T03:16:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.603+0000 7fa73321d740 1 Processor -- start 2024-09-12T03:16:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.604+0000 7fa726000640 1 -- 172.21.15.57:0/353925733 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fa720005540 con 0x7fa72810bf00 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.604+0000 7fa73321d740 1 -- start start 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.604+0000 7fa73321d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fb7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.604+0000 7fa73321d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.604+0000 7fa73321d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa73321d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598d2f1d100 con 0x5598d2fb7cd0 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa73321d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5598d2ec51b0 con 0x5598d2fc2570 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa73321d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598d2f190c0 con 0x5598d2fb73b0 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fb7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fb7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60734/0 (socket says 172.21.15.57:60734) 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72ce00640 1 -- 172.21.15.57:0/1775079941 learned_addr learned my addr 172.21.15.57:0/1775079941 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72ce00640 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 -- 172.21.15.57:0/1775079941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 msgr2=0x5598d2fb7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fb7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 -- 172.21.15.57:0/1775079941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 msgr2=0x5598d2fc4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fc4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72d800640 1 -- 172.21.15.57:0/1775079941 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d2c30210 con 0x5598d2fb7cd0 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa72ce00640 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.605+0000 7fa726000640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 0x7fa7100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa726000640 1 -- 172.21.15.57:0/353925733 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fa7200cfe80 con 0x7fa72810bf00 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa72ce00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 0x7fa7100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa72d800640 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc2030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa71c011510 tx=0x7fa71c0119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c126cdc5b086e234 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa71aa00640 1 -- 172.21.15.57:0/1775079941 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c01e070 con 0x5598d2fb7cd0 2024-09-12T03:16:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa71aa00640 1 -- 172.21.15.57:0/1775079941 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71c00faa0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.606+0000 7fa71aa00640 1 -- 172.21.15.57:0/1775079941 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c019b60 con 0x5598d2fb7cd0 2024-09-12T03:16:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 -- 172.21.15.57:0/1775079941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 msgr2=0x5598d2fc2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa72ce00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 0x7fa7100a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5598d2fb9f50 tx=0x7fa714009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc2030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa71c011510 tx=0x7fa71c0119d0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 -- 172.21.15.57:0/1775079941 shutdown_connections 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc2030 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 --2- 172.21.15.57:0/1775079941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fb7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 -- 172.21.15.57:0/1775079941 >> 172.21.15.57:0/1775079941 conn(0x5598d2fb3920 msgr2=0x5598d2fb3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 -- 172.21.15.57:0/1775079941 shutdown_connections 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.607+0000 7fa73321d740 1 -- 172.21.15.57:0/1775079941 wait complete. 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 -- start start 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 Processor -- start 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 0x5598d2fbf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 0x5598d2fbffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x5598d2fc50b0 con 0x5598d2fb73b0 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x5598d2ea6090 con 0x5598d2fb7cd0 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598d2fc5230 con 0x5598d2fc2570 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.608+0000 7fa73321d740 10 client.0 _ll_register_callbacks cb 0x5598d2efaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72d800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 0x5598d2fbf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:54960/0 (socket says 172.21.15.57:54960) 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 -- 172.21.15.57:0/970680469 learned_addr learned my addr 172.21.15.57:0/970680469 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72e200640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 0x5598d2fbffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 -- 172.21.15.57:0/970680469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 msgr2=0x5598d2fbffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 0x5598d2fbffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 -- 172.21.15.57:0/970680469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 msgr2=0x5598d2fbf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 0x5598d2fbf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 -- 172.21.15.57:0/970680469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d2cfc620 con 0x5598d2fb7cd0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72e200640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 0x5598d2fbffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72d800640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 0x5598d2fbf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa72ce00640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa71402a9a0 tx=0x7fa714005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0253c2a58909a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.609+0000 7fa70f400640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71404e070 con 0x5598d2fb7cd0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa71c00faa0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fa7140325c0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7140055d0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa73321d740 10 client.26024 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598d2fd0e30 con 0x5598d2fb7cd0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.610+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71c00faa0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa71403da60 con 0x5598d2fb7cd0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 10 client.26024 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 10 client.26024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 20 client.26024 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d2fd22a0 0x5598d2fd4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 4 client.26024 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598d2f2a830 con 0x5598d2fd22a0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.611+0000 7fa72d800640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d2fd22a0 0x5598d2fd4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.612+0000 7fa72d800640 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d2fd22a0 0x5598d2fd4690 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4990c274579247b server_cookie=47835475602f98c5 in_seq=0 out_seq=0 2024-09-12T03:16:30.617 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.612+0000 7fa70f400640 10 client.26024 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa70f400640 1 -- 172.21.15.57:0/970680469 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598d2f2a830 con 0x5598d2fd22a0 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa70f400640 10 client.26024 handle_command_reply: tid=0 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 client.26024 shutdown 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 msgr2=0x5598d2fbfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa71402a9a0 tx=0x7fa714005b50 comp rx=0 tx=0).stop 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 shutdown_connections 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d2fd22a0 0x5598d2fd4690 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fc2570 0x5598d2fbffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fb7cd0 0x5598d2fbfa80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 --2- 172.21.15.57:0/970680469 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb73b0 0x5598d2fbf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa70f400640 0 client.26024 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:30.618 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 >> 172.21.15.57:0/970680469 conn(0x5598d2fb3920 msgr2=0x5598d2fb9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 shutdown_connections 2024-09-12T03:16:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.613+0000 7fa73321d740 1 -- 172.21.15.57:0/970680469 wait complete. 2024-09-12T03:16:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.614+0000 7fa73321d740 20 client.26024 trim_cache size 0 max 16384 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 Processor -- start 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 -- start start 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fd9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fbf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281b3a20 con 0x5598d2fb7cd0 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281b12f0 con 0x5598d2fc2570 2024-09-12T03:16:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa73321d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281af160 con 0x5598d2fb73b0 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:54908/0 (socket says 172.21.15.57:54908) 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.630+0000 7fa72ce00640 1 -- 172.21.15.57:0/34274524 learned_addr learned my addr 172.21.15.57:0/34274524 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72e200640 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fbf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72d800640 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fd9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 -- 172.21.15.57:0/34274524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 msgr2=0x5598d2fd9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fd9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 -- 172.21.15.57:0/34274524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 msgr2=0x5598d2fbf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fbf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 -- 172.21.15.57:0/34274524 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d2fb7b00 con 0x5598d2fb7cd0 2024-09-12T03:16:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa72ce00640 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa714009230 tx=0x7fa714005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e889e223c69be72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa71a000640 1 -- 172.21.15.57:0/34274524 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714062070 con 0x5598d2fb7cd0 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.631+0000 7fa71a000640 1 -- 172.21.15.57:0/34274524 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa714003f40 con 0x5598d2fb7cd0 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa71a000640 1 -- 172.21.15.57:0/34274524 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71404e040 con 0x5598d2fb7cd0 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 -- 172.21.15.57:0/34274524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 msgr2=0x5598d2fda4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa714009230 tx=0x7fa714005d50 comp rx=0 tx=0).stop 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 -- 172.21.15.57:0/34274524 shutdown_connections 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d2fbf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fda4e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 --2- 172.21.15.57:0/34274524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fd9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 -- 172.21.15.57:0/34274524 >> 172.21.15.57:0/34274524 conn(0x5598d2fb3920 msgr2=0x5598d2fba7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 -- 172.21.15.57:0/34274524 shutdown_connections 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.632+0000 7fa73321d740 1 -- 172.21.15.57:0/34274524 wait complete. 2024-09-12T03:16:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 -- start start 2024-09-12T03:16:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 Processor -- start 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d3034f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281af7b0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281b0c20 con 0x5598d2fc2570 2024-09-12T03:16:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7281afe80 con 0x5598d2fb73b0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.633+0000 7fa73321d740 10 client.0 _ll_register_callbacks cb 0x7fa714009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:60762/0 (socket says 172.21.15.57:60762) 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72d800640 1 -- 172.21.15.57:0/3005537848 learned_addr learned my addr 172.21.15.57:0/3005537848 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72e200640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d3034f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 -- 172.21.15.57:0/3005537848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 msgr2=0x5598d2fc5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 -- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 msgr2=0x5598d3034f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d3034f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 -- 172.21.15.57:0/3005537848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d2fd4a00 con 0x5598d2fb7cd0 2024-09-12T03:16:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72e200640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d3034f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72d800640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa72ce00640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc6160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa71402a9a0 tx=0x7fa71403d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6de87669baa99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.634+0000 7fa70f400640 10 client.0 ms_handle_connect on v2:172.21.15.57:3300/0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71404e070 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mon.0 v2:172.21.15.57:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7fa71403d5c0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa714003f40 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa714074050 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 10 client.15885 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598d303c6f0 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mon.0 v2:172.21.15.57:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa714003f40 con 0x5598d2fb7cd0 2024-09-12T03:16:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mon.0 v2:172.21.15.57:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa714032b00 con 0x5598d2fb7cd0 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 20 client.15885 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d303cd70 0x5598d303f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 4 client.15885 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.635+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598d2f24b80 con 0x5598d303cd70 2024-09-12T03:16:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.636+0000 7fa72d800640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d303cd70 0x5598d303f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.636+0000 7fa72d800640 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d303cd70 0x5598d303f190 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf4c4d2157a63be server_cookie=d22caa869c2f8674 in_seq=0 out_seq=0 2024-09-12T03:16:30.643 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.636+0000 7fa70f400640 10 client.15885 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.640+0000 7fa70f400640 1 -- 172.21.15.57:0/3005537848 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598d2f24b80 con 0x5598d303cd70 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.640+0000 7fa70f400640 10 client.15885 handle_command_reply: tid=0 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.640+0000 7fa73321d740 1 client.15885 shutdown 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 msgr2=0x5598d2fc6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc6160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa71402a9a0 tx=0x7fa71403d450 comp rx=0 tx=0).stop 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 shutdown_connections 2024-09-12T03:16:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x5598d303cd70 0x5598d303f190 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5598d2fc2570 0x5598d3034f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x5598d2fb7cd0 0x5598d2fc6160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa70f400640 0 client.15885 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 --2- 172.21.15.57:0/3005537848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598d2fb73b0 0x5598d2fc5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 >> 172.21.15.57:0/3005537848 conn(0x5598d2fb3920 msgr2=0x5598d2fb8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 shutdown_connections 2024-09-12T03:16:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 1 -- 172.21.15.57:0/3005537848 wait complete. 2024-09-12T03:16:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.641+0000 7fa73321d740 20 client.15885 trim_cache size 0 max 16384 2024-09-12T03:16:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:16:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:16:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:16:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 msgr2=0x7fa7100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 0x7fa7100a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5598d2fb9f50 tx=0x7fa714009260 comp rx=0 tx=0).stop 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 msgr2=0x7fa72819ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72819ecf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa720009fd0 tx=0x7fa72000ef10 comp rx=0 tx=0).stop 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 shutdown_connections 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa72810bf00 0x7fa72819ecf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa728103400 0x7fa72819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fa7100a2d70 0x7fa7100a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 --2- 172.21.15.57:0/353925733 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fa728102ae0 0x7fa72819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 >> 172.21.15.57:0/353925733 conn(0x7fa7280777c0 msgr2=0x7fa7280fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 shutdown_connections 2024-09-12T03:16:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:30.642+0000 7fa72ec00640 1 -- 172.21.15.57:0/353925733 wait complete. 2024-09-12T03:16:32.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:31 smithi057 ceph-mon[30361]: pgmap v330: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:32.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:31 smithi204 ceph-mon[35013]: pgmap v330: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:32.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:31 smithi181 ceph-mon[35320]: pgmap v330: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:34.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:33 smithi057 ceph-mon[30361]: pgmap v331: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:16:34.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:33 smithi204 ceph-mon[35013]: pgmap v331: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:16:34.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:33 smithi181 ceph-mon[35320]: pgmap v331: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-12T03:16:36.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:35 smithi057 ceph-mon[30361]: pgmap v332: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:36.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:35 smithi204 ceph-mon[35013]: pgmap v332: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:36.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:35 smithi181 ceph-mon[35320]: pgmap v332: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:38.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:37 smithi057 ceph-mon[30361]: pgmap v333: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:38.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:37 smithi204 ceph-mon[35013]: pgmap v333: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:38.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:37 smithi181 ceph-mon[35320]: pgmap v333: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:40.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:39 smithi057 ceph-mon[30361]: pgmap v334: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:40.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:39 smithi204 ceph-mon[35013]: pgmap v334: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:40.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:39 smithi181 ceph-mon[35320]: pgmap v334: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:42.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:41 smithi057 ceph-mon[30361]: pgmap v335: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:42.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:41 smithi204 ceph-mon[35013]: pgmap v335: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:42.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:41 smithi181 ceph-mon[35320]: pgmap v335: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 -- 172.21.15.57:0/340240128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf81048e0 msgr2=0x7f7cf8108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 --2- 172.21.15.57:0/340240128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf81048e0 0x7f7cf8108920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009900 tx=0x7f7ce402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 -- 172.21.15.57:0/340240128 shutdown_connections 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 --2- 172.21.15.57:0/340240128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 0x7f7cf8110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 --2- 172.21.15.57:0/340240128 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf81048e0 0x7f7cf8108920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 --2- 172.21.15.57:0/340240128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf8103f30 0x7f7cf8104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 -- 172.21.15.57:0/340240128 >> 172.21.15.57:0/340240128 conn(0x7f7cf80ff980 msgr2=0x7f7cf8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 -- 172.21.15.57:0/340240128 shutdown_connections 2024-09-12T03:16:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.769+0000 7f7cfce00640 1 -- 172.21.15.57:0/340240128 wait complete. 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.770+0000 7f7cfce00640 1 Processor -- start 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.770+0000 7f7cfce00640 1 -- start start 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 0x7f7cf819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf819ee10 0x7f7cf819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81140a0 con 0x7f7cf8103f30 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8113f20 con 0x7f7cf819ee10 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cfce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8114220 con 0x7f7cf8109080 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49006/0 (socket says 172.21.15.57:49006) 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 -- 172.21.15.57:0/2067264057 learned_addr learned my addr 172.21.15.57:0/2067264057 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5400640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 0x7f7cf819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf6800640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf819ee10 0x7f7cf819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 -- 172.21.15.57:0/2067264057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 msgr2=0x7f7cf819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 0x7f7cf819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 -- 172.21.15.57:0/2067264057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf819ee10 msgr2=0x7f7cf819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.771+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf819ee10 0x7f7cf819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7cf5e00640 1 -- 172.21.15.57:0/2067264057 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf819f900 con 0x7f7cf8103f30 2024-09-12T03:16:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ea60 tx=0x7f7ce800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7ceea00640 1 -- 172.21.15.57:0/2067264057 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800cde0 con 0x7f7cf8103f30 2024-09-12T03:16:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7ceea00640 1 -- 172.21.15.57:0/2067264057 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce80035f0 con 0x7f7cf8103f30 2024-09-12T03:16:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf819fbf0 con 0x7f7cf8103f30 2024-09-12T03:16:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7ceea00640 1 -- 172.21.15.57:0/2067264057 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8010670 con 0x7f7cf8103f30 2024-09-12T03:16:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.772+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf81a7400 con 0x7f7cf8103f30 2024-09-12T03:16:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.775+0000 7f7cffc62740 1 Processor -- start 2024-09-12T03:16:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.775+0000 7f7ceea00640 1 -- 172.21.15.57:0/2067264057 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f7ce80037a0 con 0x7f7cf8103f30 2024-09-12T03:16:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 -- start start 2024-09-12T03:16:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bded9fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bded959100 con 0x55bded9fe3a0 2024-09-12T03:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bded9011b0 con 0x55bded9f31e0 2024-09-12T03:16:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cffc62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bded9550c0 con 0x55bded9f3b00 2024-09-12T03:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf5400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda00790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49022/0 (socket says 172.21.15.57:49022) 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37826/0 (socket says 172.21.15.57:37826) 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf5400640 1 -- 172.21.15.57:0/1749203385 learned_addr learned my addr 172.21.15.57:0/1749203385 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.776+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bded9fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 -- 172.21.15.57:0/1749203385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 msgr2=0x55bded9fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bded9fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 -- 172.21.15.57:0/1749203385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 msgr2=0x55bdeda00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda00790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 -- 172.21.15.57:0/1749203385 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bded66c210 con 0x55bded9f31e0 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009f00 tx=0x7f7cf0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873b55b9f4074c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7ceea00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 0x7f7cdc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7ceea00640 1 -- 172.21.15.57:0/2067264057 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f7ce80cfb30 con 0x7f7cf8103f30 2024-09-12T03:16:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7ce3400640 1 -- 172.21.15.57:0/1749203385 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0004d20 con 0x55bded9f31e0 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.777+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 0x7f7cdc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7ce3400640 1 -- 172.21.15.57:0/1749203385 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0004050 con 0x55bded9f31e0 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7ce3400640 1 -- 172.21.15.57:0/1749203385 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0005280 con 0x55bded9f31e0 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 -- 172.21.15.57:0/1749203385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 msgr2=0x55bded9f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cf5e00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 0x7f7cdc0a52e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55bded9f05b0 tx=0x7f7ce80d35e0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009f00 tx=0x7f7cf0011810 comp rx=0 tx=0).stop 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 -- 172.21.15.57:0/1749203385 shutdown_connections 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bded9fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1749203385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9f35c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.778+0000 7f7cffc62740 1 -- 172.21.15.57:0/1749203385 >> 172.21.15.57:0/1749203385 conn(0x55bded9ef750 msgr2=0x55bded9efb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.779+0000 7f7cffc62740 1 -- 172.21.15.57:0/1749203385 shutdown_connections 2024-09-12T03:16:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.779+0000 7f7cffc62740 1 -- 172.21.15.57:0/1749203385 wait complete. 2024-09-12T03:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.779+0000 7f7cffc62740 1 -- start start 2024-09-12T03:16:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.779+0000 7f7cffc62740 1 Processor -- start 2024-09-12T03:16:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bded9fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bded9fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x55bdeda00f70 con 0x55bded9fe3a0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x55bded8e2090 con 0x55bded9f3b00 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bdeda010f0 con 0x55bded9f31e0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cffc62740 10 client.0 _ll_register_callbacks cb 0x55bded936f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:49032/0 (socket says 172.21.15.57:49032) 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf6800640 1 -- 172.21.15.57:0/1710163871 learned_addr learned my addr 172.21.15.57:0/1710163871 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf5400640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bded9fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bded9fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf5400640 1 -- 172.21.15.57:0/1710163871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 msgr2=0x55bded9fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.780+0000 7f7cf5400640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bded9fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cf5400640 1 -- 172.21.15.57:0/1710163871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 msgr2=0x55bded9fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cf5400640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cf5400640 1 -- 172.21.15.57:0/1710163871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bded738620 con 0x55bded9f3b00 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cf5400640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bded9fb800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4000c00 tx=0x7f7ce4031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b74895de1faeb55d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4035040 con 0x55bded9f3b00 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf0004050 con 0x55bded9f3b00 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f7ce4004450 con 0x55bded9f3b00 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ce404d050 con 0x55bded9f3b00 2024-09-12T03:16:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cffc62740 10 client.26042 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.781+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdeda0cb80 con 0x55bded9f3b00 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0004050 con 0x55bded9f3b00 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7ce4004c10 con 0x55bded9f3b00 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 10 client.26042 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 20 client.26042 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 4 client.26042 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.782+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bded966830 con 0x55bdeda0e040 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.783+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.783+0000 7f7cf6800640 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda10430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc902bbf8836bbc server_cookie=82f9e1d13acb8b4e in_seq=0 out_seq=0 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.783+0000 7f7cd7e00640 10 client.26042 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cd7e00640 1 -- 172.21.15.57:0/1710163871 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bded966830 con 0x55bdeda0e040 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cd7e00640 10 client.26042 handle_command_reply: tid=0 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 client.26042 shutdown 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 msgr2=0x55bded9fb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bded9fb800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4000c00 tx=0x7f7ce4031850 comp rx=0 tx=0).stop 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 shutdown_connections 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda10430 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bded9fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cd7e00640 0 client.26042 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bded9fb800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 --2- 172.21.15.57:0/1710163871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bded9fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 >> 172.21.15.57:0/1710163871 conn(0x55bded9ef750 msgr2=0x55bded9f5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 shutdown_connections 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.784+0000 7f7cffc62740 1 -- 172.21.15.57:0/1710163871 wait complete. 2024-09-12T03:16:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.785+0000 7f7cffc62740 20 client.26042 trim_cache size 0 max 16384 2024-09-12T03:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 Processor -- start 2024-09-12T03:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 -- start start 2024-09-12T03:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bdeda15cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda161f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81b3ab0 con 0x55bded9fe3a0 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81b1380 con 0x55bded9f31e0 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cffc62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81af1f0 con 0x55bded9f3b00 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37846/0 (socket says 172.21.15.57:37846) 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cf6800640 1 -- 172.21.15.57:0/3143965422 learned_addr learned my addr 172.21.15.57:0/3143965422 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cf6800640 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda161f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.801+0000 7f7cf5400640 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bdeda15cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 -- 172.21.15.57:0/3143965422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 msgr2=0x55bdeda15cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bdeda15cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 -- 172.21.15.57:0/3143965422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 msgr2=0x55bdeda161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda161f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 -- 172.21.15.57:0/3143965422 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bded9ead60 con 0x55bded9f31e0 2024-09-12T03:16:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7cf6800640 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009230 tx=0x7f7cf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166aa4528cea0ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7ce2a00640 1 -- 172.21.15.57:0/3143965422 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003d00 con 0x55bded9f31e0 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7ce2a00640 1 -- 172.21.15.57:0/3143965422 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0028930 con 0x55bded9f31e0 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.802+0000 7f7ce2a00640 1 -- 172.21.15.57:0/3143965422 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00197b0 con 0x55bded9f31e0 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- 172.21.15.57:0/3143965422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 msgr2=0x55bded9fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009230 tx=0x7f7cf0004a40 comp rx=0 tx=0).stop 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- 172.21.15.57:0/3143965422 shutdown_connections 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda161f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f3b00 0x55bdeda15cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 --2- 172.21.15.57:0/3143965422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f31e0 0x55bded9fbb70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- 172.21.15.57:0/3143965422 >> 172.21.15.57:0/3143965422 conn(0x55bded9ef750 msgr2=0x55bded9fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- 172.21.15.57:0/3143965422 shutdown_connections 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- 172.21.15.57:0/3143965422 wait complete. 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.803+0000 7f7cffc62740 1 -- start start 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 Processor -- start 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bdeda01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda76c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81af840 con 0x55bded9fe3a0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81b0cb0 con 0x55bded9f3b00 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf81aff10 con 0x55bded9f31e0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cffc62740 10 client.0 _ll_register_callbacks cb 0x7f7ce4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cf5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cf6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bdeda01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cf5400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:37848/0 (socket says 172.21.15.57:37848) 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.804+0000 7f7cf5400640 1 -- 172.21.15.57:0/2467765726 learned_addr learned my addr 172.21.15.57:0/2467765726 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf6800640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda76c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 -- 172.21.15.57:0/2467765726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 msgr2=0x55bdeda01910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bdeda01910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 -- 172.21.15.57:0/2467765726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 msgr2=0x55bdeda76c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda76c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 -- 172.21.15.57:0/2467765726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdeda771b0 con 0x55bded9f3b00 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf6800640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cf5400640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40096c0 tx=0x7f7ce4031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c53e03c5c16618 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4035040 con 0x55bded9f3b00 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f7ce40039e0 con 0x55bded9f3b00 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.805+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf0028930 con 0x55bded9f3b00 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ce4061050 con 0x55bded9f3b00 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 10 client.26048 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdeda784c0 con 0x55bded9f3b00 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cf0028930 con 0x55bded9f3b00 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7ce40046c0 con 0x55bded9f3b00 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 10 client.26048 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 10 client.26048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 20 client.26048 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 4 client.26048 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bded960b80 con 0x55bdeda0e040 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.806+0000 7f7cf6800640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.807+0000 7f7cf6800640 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda7acb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c61996145b34f978 server_cookie=2b6a0e9e5223c380 in_seq=0 out_seq=0 2024-09-12T03:16:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.807+0000 7f7cd7e00640 10 client.26048 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.811+0000 7f7cd7e00640 1 -- 172.21.15.57:0/2467765726 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bded960b80 con 0x55bdeda0e040 2024-09-12T03:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.811+0000 7f7cd7e00640 10 client.26048 handle_command_reply: tid=0 2024-09-12T03:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.811+0000 7f7cffc62740 1 client.26048 shutdown 2024-09-12T03:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.811+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 msgr2=0x55bdeda01e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.811+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40096c0 tx=0x7f7ce4031af0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 shutdown_connections 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x55bdeda0e040 0x55bdeda7acb0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x55bded9fe3a0 0x55bdeda76c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55bded9f3b00 0x55bdeda01e50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cd7e00640 0 client.26048 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 --2- 172.21.15.57:0/2467765726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bded9f31e0 0x55bdeda01910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 >> 172.21.15.57:0/2467765726 conn(0x55bded9ef750 msgr2=0x55bded9fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 shutdown_connections 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 1 -- 172.21.15.57:0/2467765726 wait complete. 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.812+0000 7f7cffc62740 20 client.26048 trim_cache size 0 max 16384 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:16:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 msgr2=0x7f7cdc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 0x7f7cdc0a52e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55bded9f05b0 tx=0x7f7ce80d35e0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 msgr2=0x7f7cf819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800ea60 tx=0x7f7ce800ef20 comp rx=0 tx=0).stop 2024-09-12T03:16:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 shutdown_connections 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cf819ee10 0x7f7cf819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cf8109080 0x7f7cf819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7cdc0a2e20 0x7f7cdc0a52e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 --2- 172.21.15.57:0/2067264057 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7cf8103f30 0x7f7cf819a550 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 >> 172.21.15.57:0/2067264057 conn(0x7f7cf80ff980 msgr2=0x7f7cf8100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 shutdown_connections 2024-09-12T03:16:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:43.813+0000 7f7cfce00640 1 -- 172.21.15.57:0/2067264057 wait complete. 2024-09-12T03:16:44.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:43 smithi057 ceph-mon[30361]: pgmap v336: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:44.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:43 smithi204 ceph-mon[35013]: pgmap v336: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:44.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:43 smithi181 ceph-mon[35320]: pgmap v336: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:46.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:45 smithi057 ceph-mon[30361]: pgmap v337: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:46.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:45 smithi204 ceph-mon[35013]: pgmap v337: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:46.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:45 smithi181 ceph-mon[35320]: pgmap v337: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:48.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:47 smithi057 ceph-mon[30361]: pgmap v338: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:48.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:47 smithi204 ceph-mon[35013]: pgmap v338: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:48.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:47 smithi181 ceph-mon[35320]: pgmap v338: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-12T03:16:50.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:49 smithi057 ceph-mon[30361]: pgmap v339: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:50.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:49 smithi204 ceph-mon[35013]: pgmap v339: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:50.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:49 smithi181 ceph-mon[35320]: pgmap v339: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:52.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:51 smithi057 ceph-mon[30361]: pgmap v340: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:52.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:51 smithi204 ceph-mon[35013]: pgmap v340: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:52.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:51 smithi181 ceph-mon[35320]: pgmap v340: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:54.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:53 smithi057 ceph-mon[30361]: pgmap v341: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:54.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:53 smithi204 ceph-mon[35013]: pgmap v341: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:54.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:53 smithi181 ceph-mon[35320]: pgmap v341: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:16:56.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:55 smithi057 ceph-mon[30361]: pgmap v342: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:55 smithi204 ceph-mon[35013]: pgmap v342: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:56.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:55 smithi181 ceph-mon[35320]: pgmap v342: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 -- 172.21.15.57:0/1324389207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94103f30 msgr2=0x7f5c94104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/1324389207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94103f30 0x7f5c94104310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009900 tx=0x7f5c8402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 -- 172.21.15.57:0/1324389207 shutdown_connections 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/1324389207 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c94109080 0x7f5c94110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/1324389207 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c941048e0 0x7f5c94108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/1324389207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94103f30 0x7f5c94104310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 -- 172.21.15.57:0/1324389207 >> 172.21.15.57:0/1324389207 conn(0x7f5c940ff980 msgr2=0x7f5c94101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 -- 172.21.15.57:0/1324389207 shutdown_connections 2024-09-12T03:16:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.944+0000 7f5c9ac00640 1 -- 172.21.15.57:0/1324389207 wait complete. 2024-09-12T03:16:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.945+0000 7f5c9ac00640 1 Processor -- start 2024-09-12T03:16:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 -- start start 2024-09-12T03:16:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 0x7f5c9419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 0x7f5c9419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c9419ee20 0x7f5c9419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.949 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941140f0 con 0x7f5c94103f30 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94113f70 con 0x7f5c9419ee20 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c9ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94114270 con 0x7f5c94109080 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 0x7f5c9419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 0x7f5c9419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40594/0 (socket says 172.21.15.57:40594) 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c93400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 0x7f5c9419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c93e00640 1 -- 172.21.15.57:0/2052240411 learned_addr learned my addr 172.21.15.57:0/2052240411 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.946+0000 7f5c98c00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c9419ee20 0x7f5c9419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 -- 172.21.15.57:0/2052240411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c9419ee20 msgr2=0x7f5c9419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c9419ee20 0x7f5c9419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 -- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 msgr2=0x7f5c9419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 0x7f5c9419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 -- 172.21.15.57:0/2052240411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9419f910 con 0x7f5c94109080 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c93400640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 0x7f5c9419aaa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9c0 tx=0x7f5c8000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.947+0000 7f5c90c00640 1 -- 172.21.15.57:0/2052240411 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000cd80 con 0x7f5c94109080 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.948+0000 7f5c90c00640 1 -- 172.21.15.57:0/2052240411 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c800035f0 con 0x7f5c94109080 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.948+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9419fc00 con 0x7f5c94109080 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.948+0000 7f5c90c00640 1 -- 172.21.15.57:0/2052240411 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80010690 con 0x7f5c94109080 2024-09-12T03:16:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.948+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941a7440 con 0x7f5c94109080 2024-09-12T03:16:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 Processor -- start 2024-09-12T03:16:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c90c00640 1 -- 172.21.15.57:0/2052240411 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f5c800037a0 con 0x7f5c94109080 2024-09-12T03:16:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 -- start start 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb5ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb5d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560f0fab6100 con 0x560f0fb501e0 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560f0fa5e1b0 con 0x560f0fb50b00 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.951+0000 7f5c9da02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f0fab20c0 con 0x560f0fb5b3a0 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40608/0 (socket says 172.21.15.57:40608) 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 -- 172.21.15.57:0/1881279982 learned_addr learned my addr 172.21.15.57:0/1881279982 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c93400640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb5d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c93e00640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb5ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 -- 172.21.15.57:0/1881279982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 msgr2=0x560f0fb5d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb5d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 -- 172.21.15.57:0/1881279982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 msgr2=0x560f0fb5ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb5ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c98c00640 1 -- 172.21.15.57:0/1881279982 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0f7c9210 con 0x560f0fb501e0 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.952+0000 7f5c93400640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c98c00640 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009f50 tx=0x7f5c8c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154e30772b67808d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c90c00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 0x7f5c780a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c90c00640 1 -- 172.21.15.57:0/2052240411 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f5c800cfb70 con 0x7f5c94109080 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c89200640 1 -- 172.21.15.57:0/1881279982 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004d20 con 0x560f0fb501e0 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c89200640 1 -- 172.21.15.57:0/1881279982 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c004050 con 0x560f0fb501e0 2024-09-12T03:16:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c89200640 1 -- 172.21.15.57:0/1881279982 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c005280 con 0x560f0fb501e0 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.953+0000 7f5c93e00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 0x7f5c780a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- 172.21.15.57:0/1881279982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 msgr2=0x560f0fb505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009f50 tx=0x7f5c8c0118d0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- 172.21.15.57:0/1881279982 shutdown_connections 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb5d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb5ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1881279982 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb505c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- 172.21.15.57:0/1881279982 >> 172.21.15.57:0/1881279982 conn(0x560f0fb4c750 msgr2=0x560f0fb4cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c93e00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 0x7f5c780a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c840096c0 tx=0x7f5c840046c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- 172.21.15.57:0/1881279982 shutdown_connections 2024-09-12T03:16:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- 172.21.15.57:0/1881279982 wait complete. 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.954+0000 7f5c9da02740 1 -- start start 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 Processor -- start 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb58300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb58d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x560f0fb5df70 con 0x560f0fb501e0 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x560f0fa3f090 con 0x560f0fb50b00 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f0fb5e0f0 con 0x560f0fb5b3a0 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c9da02740 10 client.0 _ll_register_callbacks cb 0x560f0fa93f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb58300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c93400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb58d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb58300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40624/0 (socket says 172.21.15.57:40624) 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c93400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:57564/0 (socket says 172.21.15.57:57564) 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.955+0000 7f5c93e00640 1 -- 172.21.15.57:0/916901788 learned_addr learned my addr 172.21.15.57:0/916901788 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 -- 172.21.15.57:0/916901788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 msgr2=0x560f0fb58d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb58d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 -- 172.21.15.57:0/916901788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 msgr2=0x560f0fb58300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb58300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 -- 172.21.15.57:0/916901788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0f895620 con 0x560f0fb50b00 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c98c00640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb58d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c93400640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c800cce50 tx=0x7f5c8009b580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ece35c24d2134c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c7d600640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:16:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8009ffd0 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8c004050 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.956+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f5c80003b30 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c800b9050 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 10 client.26060 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f0fb69bd0 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c004050 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5c8009b070 con 0x560f0fb50b00 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 10 client.26060 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 10 client.26060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 20 client.26060 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fb6b040 0x560f0fb6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 4 client.26060 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.957+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f0fac3830 con 0x560f0fb6b040 2024-09-12T03:16:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.958+0000 7f5c93e00640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fb6b040 0x560f0fb6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.958+0000 7f5c93e00640 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fb6b040 0x560f0fb6d430 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20f45b5a918ba63 server_cookie=43300ffd041461a6 in_seq=0 out_seq=0 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.958+0000 7f5c7d600640 10 client.26060 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c7d600640 1 -- 172.21.15.57:0/916901788 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f0fac3830 con 0x560f0fb6b040 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c7d600640 10 client.26060 handle_command_reply: tid=0 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 client.26060 shutdown 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 msgr2=0x560f0fb58840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c800cce50 tx=0x7f5c8009b580 comp rx=0 tx=0).stop 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 shutdown_connections 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fb6b040 0x560f0fb6d430 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb5b3a0 0x560f0fb58d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb58840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 --2- 172.21.15.57:0/916901788 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb501e0 0x560f0fb58300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c7d600640 0 client.26060 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 >> 172.21.15.57:0/916901788 conn(0x560f0fb4c750 msgr2=0x560f0fb524b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 shutdown_connections 2024-09-12T03:16:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 1 -- 172.21.15.57:0/916901788 wait complete. 2024-09-12T03:16:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.959+0000 7f5c9da02740 20 client.26060 trim_cache size 0 max 16384 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 Processor -- start 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 -- start start 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb52ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb4b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 0x560f0fb72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941b3bf0 con 0x560f0fb5b3a0 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941b14c0 con 0x560f0fb50b00 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c9da02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941af330 con 0x560f0fb501e0 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 0x560f0fb72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.975+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb52ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 0x560f0fb72c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:40626/0 (socket says 172.21.15.57:40626) 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c98c00640 1 -- 172.21.15.57:0/3599453358 learned_addr learned my addr 172.21.15.57:0/3599453358 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93400640 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb4b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 -- 172.21.15.57:0/3599453358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 msgr2=0x560f0fb4b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb4b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 -- 172.21.15.57:0/3599453358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 msgr2=0x560f0fb72c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 0x560f0fb72c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 -- 172.21.15.57:0/3599453358 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0fb6d9f0 con 0x560f0fb501e0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c93e00640 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb52ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c84054fd0 tx=0x7f5c8403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28e725881b42205 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.976+0000 7f5c7fe00640 1 -- 172.21.15.57:0/3599453358 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84035040 con 0x560f0fb501e0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c7fe00640 1 -- 172.21.15.57:0/3599453358 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c84031d60 con 0x560f0fb501e0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c7fe00640 1 -- 172.21.15.57:0/3599453358 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840548b0 con 0x560f0fb501e0 2024-09-12T03:16:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 -- 172.21.15.57:0/3599453358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 msgr2=0x560f0fb52ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb52ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c84054fd0 tx=0x7f5c8403f000 comp rx=0 tx=0).stop 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 -- 172.21.15.57:0/3599453358 shutdown_connections 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb5b3a0 0x560f0fb72c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb50b00 0x560f0fb4b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 --2- 172.21.15.57:0/3599453358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb52ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 -- 172.21.15.57:0/3599453358 >> 172.21.15.57:0/3599453358 conn(0x560f0fb4c750 msgr2=0x560f0fb4b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 -- 172.21.15.57:0/3599453358 shutdown_connections 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.977+0000 7f5c9da02740 1 -- 172.21.15.57:0/3599453358 wait complete. 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 -- start start 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 Processor -- start 2024-09-12T03:16:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb50b00 0x560f0fb54df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb5b3a0 0x560f0fb55330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941af980 con 0x560f0fb50b00 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941b0df0 con 0x560f0fb5b3a0 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941b0050 con 0x560f0fb501e0 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.978+0000 7f5c9da02740 10 client.0 _ll_register_callbacks cb 0x7f5c80003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:58218/0 (socket says 172.21.15.57:58218) 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 -- 172.21.15.57:0/1688724535 learned_addr learned my addr 172.21.15.57:0/1688724535 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:16:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb50b00 0x560f0fb54df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c98c00640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb5b3a0 0x560f0fb55330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 -- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb5b3a0 msgr2=0x560f0fb55330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb5b3a0 0x560f0fb55330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 -- 172.21.15.57:0/1688724535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb50b00 msgr2=0x560f0fb54df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb50b00 0x560f0fb54df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 -- 172.21.15.57:0/1688724535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0fb47d60 con 0x560f0fb501e0 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c93e00640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c840092c0 tx=0x7f5c84031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21a81dd0c216b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.979+0000 7f5c7d600640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84061020 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c84031d60 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9554+0+0 (secure 0 0 0) 0x7f5c840548c0 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5c84054ac0 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 10 client.26089 fetch_fsmap learned FSMap version 30 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f0fbd5860 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c84031d60 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5c8403d610 con 0x560f0fb501e0 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 10 client.26089 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:16:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 20 client.26089 populate_metadata read hostname 'smithi057' 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.980+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fbd5fe0 0x560f0fbd8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.981+0000 7f5c9da02740 4 client.26089 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.981+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f0fabdb80 con 0x560f0fbd5fe0 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.981+0000 7f5c93400640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fbd5fe0 0x560f0fbd8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.981+0000 7f5c93400640 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fbd5fe0 0x560f0fbd8400 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee380f978c6bbcc server_cookie=2f34f188bce75ead in_seq=0 out_seq=0 2024-09-12T03:16:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.981+0000 7f5c7d600640 10 client.26089 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.985+0000 7f5c7d600640 1 -- 172.21.15.57:0/1688724535 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f0fabdb80 con 0x560f0fbd5fe0 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.985+0000 7f5c7d600640 10 client.26089 handle_command_reply: tid=0 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.985+0000 7f5c9da02740 1 client.26089 shutdown 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 msgr2=0x560f0fb637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5c840092c0 tx=0x7f5c84031f10 comp rx=0 tx=0).stop 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 shutdown_connections 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x560f0fbd5fe0 0x560f0fbd8400 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x560f0fb5b3a0 0x560f0fb55330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c7d600640 0 client.26089 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:16:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x560f0fb50b00 0x560f0fb54df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 --2- 172.21.15.57:0/1688724535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f0fb501e0 0x560f0fb637e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 >> 172.21.15.57:0/1688724535 conn(0x560f0fb4c750 msgr2=0x560f0fb56770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 shutdown_connections 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 1 -- 172.21.15.57:0/1688724535 wait complete. 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.986+0000 7f5c9da02740 20 client.26089 trim_cache size 0 max 16384 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:16:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 msgr2=0x7f5c780a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 0x7f5c780a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c840096c0 tx=0x7f5c840046c0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 msgr2=0x7f5c9419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 0x7f5c9419aaa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9c0 tx=0x7f5c8000ee80 comp rx=0 tx=0).stop 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 shutdown_connections 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c9419ee20 0x7f5c9419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c94109080 0x7f5c9419aaa0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5c780a2e20 0x7f5c780a52e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 --2- 172.21.15.57:0/2052240411 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5c94103f30 0x7f5c9419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 >> 172.21.15.57:0/2052240411 conn(0x7f5c940ff980 msgr2=0x7f5c94101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 shutdown_connections 2024-09-12T03:16:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:16:56.987+0000 7f5c9ac00640 1 -- 172.21.15.57:0/2052240411 wait complete. 2024-09-12T03:16:58.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:57 smithi057 ceph-mon[30361]: pgmap v343: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:57 smithi204 ceph-mon[35013]: pgmap v343: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:16:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:57 smithi181 ceph-mon[35320]: pgmap v343: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-12T03:17:00.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:16:59 smithi057 ceph-mon[30361]: pgmap v344: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:17:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:16:59 smithi204 ceph-mon[35013]: pgmap v344: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:17:00.205 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:16:59 smithi181 ceph-mon[35320]: pgmap v344: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-12T03:17:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:01 smithi057 ceph-mon[30361]: pgmap v345: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:01 smithi204 ceph-mon[35013]: pgmap v345: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:01 smithi181 ceph-mon[35320]: pgmap v345: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:03 smithi057 ceph-mon[30361]: pgmap v346: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:17:04.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:03 smithi204 ceph-mon[35013]: pgmap v346: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:17:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:03 smithi181 ceph-mon[35320]: pgmap v346: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-12T03:17:04.533 INFO:tasks.workunit.client.0.smithi057.stdout:stopping iogen 2024-09-12T03:17:04.533 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo 'stopping iogen' 2024-09-12T03:17:04.533 INFO:tasks.workunit.client.0.smithi057.stderr:+ ./iogen -k 2024-09-12T03:17:04.535 INFO:tasks.workunit.client.0.smithi057.stdout:OK 2024-09-12T03:17:04.535 INFO:tasks.workunit.client.0.smithi057.stderr:+ echo OK 2024-09-12T03:17:06.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:05 smithi204 ceph-mon[35013]: pgmap v347: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:17:06.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:05 smithi181 ceph-mon[35320]: pgmap v347: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:17:06.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:05 smithi057 ceph-mon[30361]: pgmap v347: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 25 KiB/s wr, 8 op/s 2024-09-12T03:17:06.565 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:17:06.565 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:17:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:07 smithi204 ceph-mon[35013]: pgmap v348: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-09-12T03:17:08.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:07 smithi204 ceph-mon[35013]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:17:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:07 smithi181 ceph-mon[35320]: pgmap v348: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-09-12T03:17:08.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:07 smithi181 ceph-mon[35320]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:17:08.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:07 smithi057 ceph-mon[30361]: pgmap v348: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-09-12T03:17:08.371 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:07 smithi057 ceph-mon[30361]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 -- 172.21.15.57:0/305698281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010d490 msgr2=0x7f90e0114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 --2- 172.21.15.57:0/305698281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010d490 0x7f90e0114fe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009be0 tx=0x7f90d802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 -- 172.21.15.57:0/305698281 shutdown_connections 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 --2- 172.21.15.57:0/305698281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e010d490 0x7f90e0114fe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 --2- 172.21.15.57:0/305698281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e0108cf0 0x7f90e010cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 --2- 172.21.15.57:0/305698281 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90e0108340 0x7f90e0108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.184+0000 7f90e7200640 1 -- 172.21.15.57:0/305698281 >> 172.21.15.57:0/305698281 conn(0x7f90e0077a40 msgr2=0x7f90e0079e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.185+0000 7f90e7200640 1 -- 172.21.15.57:0/305698281 shutdown_connections 2024-09-12T03:17:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.185+0000 7f90e7200640 1 -- 172.21.15.57:0/305698281 wait complete. 2024-09-12T03:17:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 Processor -- start 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 -- start start 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90e010d490 0x7f90e0073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e006cad0 0x7f90e006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01184c0 con 0x7f90e010d490 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0118340 con 0x7f90e006cad0 2024-09-12T03:17:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.187+0000 7f90e7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0118640 con 0x7f90e0108340 2024-09-12T03:17:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43264/0 (socket says 172.21.15.57:43264) 2024-09-12T03:17:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 -- 172.21.15.57:0/1713293092 learned_addr learned my addr 172.21.15.57:0/1713293092 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90e5200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e006cad0 0x7f90e006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 -- 172.21.15.57:0/1713293092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e006cad0 msgr2=0x7f90e006cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e006cad0 0x7f90e006cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 -- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90e010d490 msgr2=0x7f90e0073e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90e010d490 0x7f90e0073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.188+0000 7f90dfe00640 1 -- 172.21.15.57:0/1713293092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e0071460 con 0x7f90e0108340 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.189+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00d950 tx=0x7f90cc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.190+0000 7f90dcc00640 1 -- 172.21.15.57:0/1713293092 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc017070 con 0x7f90e0108340 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.190+0000 7f90dcc00640 1 -- 172.21.15.57:0/1713293092 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90cc009c30 con 0x7f90e0108340 2024-09-12T03:17:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.190+0000 7f90dcc00640 1 -- 172.21.15.57:0/1713293092 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00f6b0 con 0x7f90e0108340 2024-09-12T03:17:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.191+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e00716f0 con 0x7f90e0108340 2024-09-12T03:17:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.191+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e0071bc0 con 0x7f90e0108340 2024-09-12T03:17:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.200+0000 7f90dcc00640 1 -- 172.21.15.57:0/1713293092 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f90cc0109d0 con 0x7f90e0108340 2024-09-12T03:17:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.201+0000 7f90e9fe4740 1 Processor -- start 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 -- start start 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d51e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556719cad100 con 0x556719d523a0 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556719c551b0 con 0x556719d471e0 2024-09-12T03:17:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.202+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556719ca90c0 con 0x556719d47b00 2024-09-12T03:17:10.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:09 smithi181 ceph-mon[35320]: pgmap v350: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90dcc00640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 0x7f90c40a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45692/0 (socket says 172.21.15.57:45692) 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 -- 172.21.15.57:0/431649594 learned_addr learned my addr 172.21.15.57:0/431649594 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90dcc00640 1 -- 172.21.15.57:0/1713293092 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7f90cc0cf5d0 con 0x7f90e0108340 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90e5200640 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 -- 172.21.15.57:0/431649594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 msgr2=0x556719d51e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d51e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 -- 172.21.15.57:0/431649594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 msgr2=0x556719d475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.203+0000 7f90df400640 1 -- 172.21.15.57:0/431649594 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567199c0210 con 0x556719d523a0 2024-09-12T03:17:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.204+0000 7f90e5200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 0x7f90c40a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.204+0000 7f90df400640 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009a20 tx=0x7f90d0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392cc9cff2388412 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.205+0000 7f90d5600640 1 -- 172.21.15.57:0/431649594 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000ecb0 con 0x556719d523a0 2024-09-12T03:17:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.205+0000 7f90e5200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 0x7f90c40a53b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90d8004af0 tx=0x7f90d802a6c0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.206+0000 7f90d5600640 1 -- 172.21.15.57:0/431649594 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d000ee50 con 0x556719d523a0 2024-09-12T03:17:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.206+0000 7f90d5600640 1 -- 172.21.15.57:0/431649594 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004ae0 con 0x556719d523a0 2024-09-12T03:17:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 -- 172.21.15.57:0/431649594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 msgr2=0x556719d54790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009a20 tx=0x7f90d0009ee0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 -- 172.21.15.57:0/431649594 shutdown_connections 2024-09-12T03:17:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d523a0 0x556719d54790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d51e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/431649594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.207+0000 7f90e9fe4740 1 -- 172.21.15.57:0/431649594 >> 172.21.15.57:0/431649594 conn(0x556719d43750 msgr2=0x556719d43b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.208+0000 7f90e9fe4740 1 -- 172.21.15.57:0/431649594 shutdown_connections 2024-09-12T03:17:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.208+0000 7f90e9fe4740 1 -- 172.21.15.57:0/431649594 wait complete. 2024-09-12T03:17:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.209+0000 7f90e9fe4740 1 -- start start 2024-09-12T03:17:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.209+0000 7f90e9fe4740 1 Processor -- start 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x556719d65fd0 con 0x556719d47b00 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x556719c36090 con 0x556719d471e0 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556719d66150 con 0x556719d523a0 2024-09-12T03:17:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e9fe4740 10 client.0 _ll_register_callbacks cb 0x556719c8af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.210+0000 7f90e5200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43278/0 (socket says 172.21.15.57:43278) 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 -- 172.21.15.57:0/1609060938 learned_addr learned my addr 172.21.15.57:0/1609060938 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90df400640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 -- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 msgr2=0x556719d4f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d4f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 -- 172.21.15.57:0/1609060938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 msgr2=0x556719d4f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d4f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90e5200640 1 -- 172.21.15.57:0/1609060938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556719a8c620 con 0x556719d523a0 2024-09-12T03:17:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.211+0000 7f90df400640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.213+0000 7f90e5200640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90d8033000 tx=0x7f90d8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20d7ce53ed30334 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.213+0000 7f90ca000640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.213+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8048040 con 0x556719d523a0 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.213+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d000ee50 con 0x556719d523a0 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.214+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7f90d8004240 con 0x556719d523a0 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.214+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90d805a050 con 0x556719d523a0 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.214+0000 7f90e9fe4740 10 client.26098 fetch_fsmap learned FSMap version 30 2024-09-12T03:17:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.214+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556719d71bb0 con 0x556719d523a0 2024-09-12T03:17:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.214+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d000ee50 con 0x556719d523a0 2024-09-12T03:17:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.217+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90d804b030 con 0x556719d523a0 2024-09-12T03:17:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 10 client.26098 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 20 client.26098 populate_metadata read hostname 'smithi057' 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719d73020 0x556719d75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 4 client.26098 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556719cba830 con 0x556719d73020 2024-09-12T03:17:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.218+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719d73020 0x556719d75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.219+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719d73020 0x556719d75470 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4702a95ce37099ac server_cookie=69c937b28bac25da in_seq=0 out_seq=0 2024-09-12T03:17:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.219+0000 7f90ca000640 10 client.26098 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.220+0000 7f90ca000640 1 -- 172.21.15.57:0/1609060938 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556719cba830 con 0x556719d73020 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90ca000640 10 client.26098 handle_command_reply: tid=0 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 client.26098 shutdown 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 msgr2=0x556719d4fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90d8033000 tx=0x7f90d8009500 comp rx=0 tx=0).stop 2024-09-12T03:17:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 shutdown_connections 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719d73020 0x556719d75470 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90ca000640 0 client.26098 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d4fc30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1609060938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d4f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.221+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 >> 172.21.15.57:0/1609060938 conn(0x556719d43750 msgr2=0x556719d49460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.222+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 shutdown_connections 2024-09-12T03:17:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.222+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1609060938 wait complete. 2024-09-12T03:17:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.222+0000 7f90e9fe4740 20 client.26098 trim_cache size 0 max 16384 2024-09-12T03:17:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 Processor -- start 2024-09-12T03:17:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 -- start start 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d471e0 0x556719d7add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 0x556719d42010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01b8180 con 0x556719d471e0 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01b5a50 con 0x556719d523a0 2024-09-12T03:17:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90e006d810 con 0x556719d47b00 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.254+0000 7f90df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90df400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43282/0 (socket says 172.21.15.57:43282) 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90df400640 1 -- 172.21.15.57:0/1955193407 learned_addr learned my addr 172.21.15.57:0/1955193407 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90e5200640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 0x556719d42010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90df400640 1 -- 172.21.15.57:0/1955193407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 msgr2=0x556719d42010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90df400640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 0x556719d42010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d471e0 0x556719d7add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.255+0000 7f90df400640 1 -- 172.21.15.57:0/1955193407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d471e0 msgr2=0x556719d7add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.256+0000 7f90df400640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d471e0 0x556719d7add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.256+0000 7f90df400640 1 -- 172.21.15.57:0/1955193407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556719d43580 con 0x556719d47b00 2024-09-12T03:17:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.257+0000 7f90e5200640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 0x556719d42010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.257+0000 7f90df400640 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006010 tx=0x7f90d0004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7244813d95f191bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.257+0000 7f90d4c00640 1 -- 172.21.15.57:0/1955193407 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0028070 con 0x556719d47b00 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.257+0000 7f90d4c00640 1 -- 172.21.15.57:0/1955193407 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d000ebb0 con 0x556719d47b00 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.258+0000 7f90d4c00640 1 -- 172.21.15.57:0/1955193407 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0028070 con 0x556719d47b00 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1955193407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 msgr2=0x556719d7b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006010 tx=0x7f90d0004600 comp rx=0 tx=0).stop 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1955193407 shutdown_connections 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d523a0 0x556719d42010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d47b00 0x556719d7b310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1955193407 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d471e0 0x556719d7add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1955193407 >> 172.21.15.57:0/1955193407 conn(0x556719d43750 msgr2=0x556719d4e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1955193407 shutdown_connections 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.259+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1955193407 wait complete. 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.260+0000 7f90e9fe4740 1 -- start start 2024-09-12T03:17:10.261 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.260+0000 7f90e9fe4740 1 Processor -- start 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d66d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0082920 con 0x556719d47b00 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01b5380 con 0x556719d471e0 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0082fd0 con 0x556719d523a0 2024-09-12T03:17:10.262 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.261+0000 7f90e9fe4740 10 client.0 _ll_register_callbacks cb 0x7f90d80306d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48448/0 (socket says 172.21.15.57:48448) 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90dfe00640 1 -- 172.21.15.57:0/1441745852 learned_addr learned my addr 172.21.15.57:0/1441745852 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90df400640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90e5200640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d66d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.262+0000 7f90dfe00640 1 -- 172.21.15.57:0/1441745852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 msgr2=0x556719d66d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d66d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90dfe00640 1 -- 172.21.15.57:0/1441745852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 msgr2=0x556719d6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90dfe00640 1 -- 172.21.15.57:0/1441745852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556719d67580 con 0x556719d471e0 2024-09-12T03:17:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90df400640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90dfe00640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f90cc09b330 tx=0x7f90cc010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd55b855181edf36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.263+0000 7f90ca000640 10 client.0 ms_handle_connect on v2:172.21.15.181:3300/0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc09b630 con 0x556719d471e0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mon.2 v2:172.21.15.181:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7f90cc097ca0 con 0x556719d471e0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d000ebb0 con 0x556719d471e0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90cc09f9e0 con 0x556719d471e0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90e9fe4740 10 client.26078 fetch_fsmap learned FSMap version 30 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.264+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556719ddda20 con 0x556719d471e0 2024-09-12T03:17:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.265+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mon.2 v2:172.21.15.181:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d000ebb0 con 0x556719d471e0 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.266+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mon.2 v2:172.21.15.181:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90cc023060 con 0x556719d471e0 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.266+0000 7f90e9fe4740 10 client.26078 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.266+0000 7f90e9fe4740 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.266+0000 7f90e9fe4740 20 client.26078 populate_metadata read hostname 'smithi057' 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.267+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719dde1e0 0x556719de05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.267+0000 7f90e9fe4740 4 client.26078 mds_command: new command op to 24554 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T03:17:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.267+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 --> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556719cb4b80 con 0x556719dde1e0 2024-09-12T03:17:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.267+0000 7f90df400640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719dde1e0 0x556719de05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.268+0000 7f90df400640 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719dde1e0 0x556719de05d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1564251dedf16c44 server_cookie=9f4f8ab7c0fcb025 in_seq=0 out_seq=0 2024-09-12T03:17:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.268+0000 7f90ca000640 10 client.26078 ms_handle_connect on v2:172.21.15.181:6838/399579254 2024-09-12T03:17:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.274+0000 7f90ca000640 1 -- 172.21.15.57:0/1441745852 <== mds.0 v2:172.21.15.181:6838/399579254 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556719cb4b80 con 0x556719dde1e0 2024-09-12T03:17:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.274+0000 7f90ca000640 10 client.26078 handle_command_reply: tid=0 2024-09-12T03:17:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 client.26078 shutdown 2024-09-12T03:17:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 msgr2=0x556719d6b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f90cc09b330 tx=0x7f90cc010d70 comp rx=0 tx=0).stop 2024-09-12T03:17:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 shutdown_connections 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:6838/399579254,v1:172.21.15.181:6839/399579254] conn(0x556719dde1e0 0x556719de05d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556719d523a0 0x556719d66d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x556719d47b00 0x556719d6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 --2- 172.21.15.57:0/1441745852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556719d471e0 0x556719d6b790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.275+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 >> 172.21.15.57:0/1441745852 conn(0x556719d43750 msgr2=0x556719d50820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.276+0000 7f90ca000640 0 client.26078 ms_handle_reset on v2:172.21.15.181:6838/399579254 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.276+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 shutdown_connections 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.276+0000 7f90e9fe4740 1 -- 172.21.15.57:0/1441745852 wait complete. 2024-09-12T03:17:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.276+0000 7f90e9fe4740 20 client.26078 trim_cache size 0 max 16384 2024-09-12T03:17:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:{ 2024-09-12T03:17:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "message": "", 2024-09-12T03:17:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout: "return_code": 0 2024-09-12T03:17:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stdout:} 2024-09-12T03:17:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.278+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 msgr2=0x7f90c40a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.278+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 0x7f90c40a53b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f90d8004af0 tx=0x7f90d802a6c0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.278+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 msgr2=0x7f90e0073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.278+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00d950 tx=0x7f90cc00de10 comp rx=0 tx=0).stop 2024-09-12T03:17:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 shutdown_connections 2024-09-12T03:17:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90e006cad0 0x7f90e006cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f90c40a2ef0 0x7f90c40a53b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f90e010d490 0x7f90e0073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 --2- 172.21.15.57:0/1713293092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0108340 0x7f90e0073940 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 >> 172.21.15.57:0/1713293092 conn(0x7f90e0077a40 msgr2=0x7f90e0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 shutdown_connections 2024-09-12T03:17:10.281 INFO:teuthology.task.background_exec.ubuntu@smithi057.front.sepia.ceph.com.smithi057.stderr:2024-09-12T03:17:10.280+0000 7f90e7200640 1 -- 172.21.15.57:0/1713293092 wait complete. 2024-09-12T03:17:10.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:09 smithi057 ceph-mon[30361]: pgmap v350: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:10.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:09 smithi204 ceph-mon[35013]: pgmap v350: 129 pgs: 129 active+clean; 18 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:10.728 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-12T03:17:10.729 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:17:12.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:11 smithi181 ceph-mon[35320]: pgmap v351: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:12.372 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:11 smithi057 ceph-mon[30361]: pgmap v351: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:12.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:11 smithi204 ceph-mon[35013]: pgmap v351: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 17 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 73 KiB/s wr, 27 op/s 2024-09-12T03:17:13.285 DEBUG:teuthology.parallel:result is None 2024-09-12T03:17:13.286 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:17:13.411 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:17:13.412 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T03:17:13.423 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:17:13.444 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T03:17:13.468 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T03:17:13.468 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:17:13.468 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:17:13.469 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T03:17:13.621 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T03:17:13.745 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 -- 172.21.15.57:0/1538024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 msgr2=0x7f8294107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 --2- 172.21.15.57:0/1538024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f8294107f60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8284009c10 tx=0x7f828402a700 comp rx=0 tx=0).stop 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 -- 172.21.15.57:0/1538024 shutdown_connections 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 --2- 172.21.15.57:0/1538024 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8294102480 0x7f829410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 --2- 172.21.15.57:0/1538024 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 0x7f8294101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 --2- 172.21.15.57:0/1538024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f8294107f60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 -- 172.21.15.57:0/1538024 >> 172.21.15.57:0/1538024 conn(0x7f82940fd7e0 msgr2=0x7f82940ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 -- 172.21.15.57:0/1538024 shutdown_connections 2024-09-12T03:17:14.039 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.038+0000 7f829b200640 1 -- 172.21.15.57:0/1538024 wait complete. 2024-09-12T03:17:14.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.040+0000 7f829b200640 1 Processor -- start 2024-09-12T03:17:14.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.042+0000 7f829b200640 1 -- start start 2024-09-12T03:17:14.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.043+0000 7f829b200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 0x7f8294198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.043+0000 7f829b200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8294102480 0x7f82941988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.043+0000 7f8293e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 0x7f8294198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.044+0000 7f829b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.044+0000 7f829b200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8294111ef0 con 0x7f8294101ae0 2024-09-12T03:17:14.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.044+0000 7f829b200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8294111d70 con 0x7f8294102480 2024-09-12T03:17:14.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.044+0000 7f829b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8294112070 con 0x7f8294107b80 2024-09-12T03:17:14.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.045 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43312/0 (socket says 172.21.15.57:43312) 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 -- 172.21.15.57:0/734519043 learned_addr learned my addr 172.21.15.57:0/734519043 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 -- 172.21.15.57:0/734519043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8294102480 msgr2=0x7f82941988c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8294102480 0x7f82941988c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 -- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 msgr2=0x7f8294198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 0x7f8294198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.046 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.045+0000 7f8299200640 1 -- 172.21.15.57:0/734519043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829419d370 con 0x7f8294107b80 2024-09-12T03:17:14.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f8299200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f828c00e970 tx=0x7f828c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.047 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c00ccc0 con 0x7f8294107b80 2024-09-12T03:17:14.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828c005500 con 0x7f8294107b80 2024-09-12T03:17:14.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c010690 con 0x7f8294107b80 2024-09-12T03:17:14.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829419d660 con 0x7f8294107b80 2024-09-12T03:17:14.048 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.046+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8294074420 con 0x7f8294107b80 2024-09-12T03:17:14.049 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.049+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8294101f40 con 0x7f8294107b80 2024-09-12T03:17:14.052 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.051+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f828c004e10 con 0x7f8294107b80 2024-09-12T03:17:14.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.055+0000 7f8291600640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 0x7f82780a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.056+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7f828c0cfba0 con 0x7f8294107b80 2024-09-12T03:17:14.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.056+0000 7f8293e00640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 0x7f82780a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.057+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f828c09c360 con 0x7f8294107b80 2024-09-12T03:17:14.060 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.057+0000 7f8293e00640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 0x7f82780a96f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8284009370 tx=0x7f8284005fd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.121 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:13 smithi057 ceph-mon[30361]: pgmap v352: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:14.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:13 smithi181 ceph-mon[35320]: pgmap v352: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:14.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:13 smithi204 ceph-mon[35013]: pgmap v352: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:14.432 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.432+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f82940fe600 con 0x7f82780a7230 2024-09-12T03:17:14.482 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.481+0000 7f8291600640 1 -- 172.21.15.57:0/734519043 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f82940fe600 con 0x7f82780a7230 2024-09-12T03:17:14.484 INFO:teuthology.orchestra.run.smithi057.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T03:05:58", "first": "2024-09-12T03:06:00", "last": "2024-09-12T03:17:00", "last_pruned": "2024-09-12T03:17:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-09-12T03:17:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.485+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 msgr2=0x7f82780a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.485+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 0x7f82780a96f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8284009370 tx=0x7f8284005fd0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.485+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 msgr2=0x7f829419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.485 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.485+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f828c00e970 tx=0x7f828c00ee30 comp rx=0 tx=0).stop 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 shutdown_connections 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294107b80 0x7f829419cc40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8294102480 0x7f82941988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f82780a7230 0x7f82780a96f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 --2- 172.21.15.57:0/734519043 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8294101ae0 0x7f8294198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 >> 172.21.15.57:0/734519043 conn(0x7f82940fd7e0 msgr2=0x7f82940fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 shutdown_connections 2024-09-12T03:17:14.487 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.486+0000 7f829b200640 1 -- 172.21.15.57:0/734519043 wait complete. 2024-09-12T03:17:14.513 DEBUG:teuthology.orchestra.run.smithi057:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T03:17:14.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.783+0000 7fcdd7200640 1 -- 172.21.15.57:0/3026008715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0109080 msgr2=0x7fcdd0110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.783+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3026008715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0109080 0x7fcdd0110bd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009ca0 tx=0x7fcdc802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.785+0000 7fcdd7200640 1 -- 172.21.15.57:0/3026008715 shutdown_connections 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.785+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3026008715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0109080 0x7fcdd0110bd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.785+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3026008715 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd01048e0 0x7fcdd0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.785+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3026008715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0103f30 0x7fcdd0104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.785+0000 7fcdd7200640 1 -- 172.21.15.57:0/3026008715 >> 172.21.15.57:0/3026008715 conn(0x7fcdd00ff980 msgr2=0x7fcdd0101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.786 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.786+0000 7fcdd7200640 1 -- 172.21.15.57:0/3026008715 shutdown_connections 2024-09-12T03:17:14.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.786+0000 7fcdd7200640 1 -- 172.21.15.57:0/3026008715 wait complete. 2024-09-12T03:17:14.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.787+0000 7fcdd7200640 1 Processor -- start 2024-09-12T03:17:14.788 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.788+0000 7fcdd7200640 1 -- start start 2024-09-12T03:17:14.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.789+0000 7fcdd7200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.789 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.789+0000 7fcdd7200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0109080 0x7fcdd0073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdd7200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 0x7fcdd010fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdd7200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0077590 con 0x7fcdd010f710 2024-09-12T03:17:14.790 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdd7200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0077410 con 0x7fcdd0109080 2024-09-12T03:17:14.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdd7200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd0077710 con 0x7fcdd0103f30 2024-09-12T03:17:14.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdcfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43320/0 (socket says 172.21.15.57:43320) 2024-09-12T03:17:14.791 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdcfe00640 1 -- 172.21.15.57:0/3331483375 learned_addr learned my addr 172.21.15.57:0/3331483375 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.790+0000 7fcdd5200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 0x7fcdd010fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdcf400640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0109080 0x7fcdd0073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 -- 172.21.15.57:0/3331483375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 msgr2=0x7fcdd010f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 -- 172.21.15.57:0/3331483375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0109080 msgr2=0x7fcdd0073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.792 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0109080 0x7fcdd0073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 -- 172.21.15.57:0/3331483375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd01b38d0 con 0x7fcdd010f710 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdcfe00640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.791+0000 7fcdd5200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 0x7fcdd010fbc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcdc802ae00 tx=0x7fcdc8032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.792+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8003b70 con 0x7fcdd010f710 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.792+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc8003d10 con 0x7fcdd010f710 2024-09-12T03:17:14.793 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.792+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd01b3b60 con 0x7fcdd010f710 2024-09-12T03:17:14.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.792+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8004720 con 0x7fcdd010f710 2024-09-12T03:17:14.794 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.793+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01b4100 con 0x7fcdd010f710 2024-09-12T03:17:14.795 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.795+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd0104310 con 0x7fcdd010f710 2024-09-12T03:17:14.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.799+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fcdc8030880 con 0x7fcdd010f710 2024-09-12T03:17:14.803 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.803+0000 7fcdccc00640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 0x7fcdb40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.804 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.803+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7fcdc80f0980 con 0x7fcdd010f710 2024-09-12T03:17:14.805 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.805+0000 7fcdcfe00640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 0x7fcdb40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.807 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.807+0000 7fcdcfe00640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 0x7fcdb40a5310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e00 tx=0x7fcdc0009190 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.809 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:14.808+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdc80bd250 con 0x7fcdd010f710 2024-09-12T03:17:15.196 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.196+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 --> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fcdd01049c0 con 0x7fcdb40a2e50 2024-09-12T03:17:15.267 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.267+0000 7fcdccc00640 1 -- 172.21.15.57:0/3331483375 <== mgr.15126 v2:172.21.15.57:6800/260348644 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fcdd01049c0 con 0x7fcdb40a2e50 2024-09-12T03:17:15.269 INFO:teuthology.orchestra.run.smithi057.stdout:/ 1m 6m3h 2024-09-12T03:17:15.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 msgr2=0x7fcdb40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.273 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 0x7fcdb40a5310 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e00 tx=0x7fcdc0009190 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 msgr2=0x7fcdd010fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 0x7fcdd010fbc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcdc802ae00 tx=0x7fcdc8032fb0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 shutdown_connections 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fcdb40a2e50 0x7fcdb40a5310 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009e00 tx=0x7fcdc0009190 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fcdd010f710 0x7fcdd010fbc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdd0109080 0x7fcdd0073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 --2- 172.21.15.57:0/3331483375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcdd0103f30 0x7fcdd010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.274 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.273+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 >> 172.21.15.57:0/3331483375 conn(0x7fcdd00ff980 msgr2=0x7fcdd0101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.274+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 shutdown_connections 2024-09-12T03:17:15.275 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.274+0000 7fcdd7200640 1 -- 172.21.15.57:0/3331483375 wait complete. 2024-09-12T03:17:15.300 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:17:15.308 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.580+0000 7ff4bb200640 1 -- 172.21.15.57:0/835532233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc109b80 msgr2=0x7ff4bc109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.580+0000 7ff4bb200640 1 --2- 172.21.15.57:0/835532233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc109b80 0x7ff4bc109f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009930 tx=0x7ff4ac02a440 comp rx=0 tx=0).stop 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 -- 172.21.15.57:0/835532233 shutdown_connections 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 --2- 172.21.15.57:0/835532233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc104480 0x7ff4bc1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 --2- 172.21.15.57:0/835532233 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 0x7ff4bc103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.581 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 --2- 172.21.15.57:0/835532233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc109b80 0x7ff4bc109f60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 -- 172.21.15.57:0/835532233 >> 172.21.15.57:0/835532233 conn(0x7ff4bc0ff7c0 msgr2=0x7ff4bc101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 -- 172.21.15.57:0/835532233 shutdown_connections 2024-09-12T03:17:15.582 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.581+0000 7ff4bb200640 1 -- 172.21.15.57:0/835532233 wait complete. 2024-09-12T03:17:15.583 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.583+0000 7ff4bb200640 1 Processor -- start 2024-09-12T03:17:15.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.583+0000 7ff4bb200640 1 -- start start 2024-09-12T03:17:15.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 0x7ff4bc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc104480 0x7ff4bc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.584 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc113f80 con 0x7ff4bc103ae0 2024-09-12T03:17:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc113e00 con 0x7ff4bc109b80 2024-09-12T03:17:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4bb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4bc114100 con 0x7ff4bc104480 2024-09-12T03:17:15.585 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4ba800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4ba800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:48490/0 (socket says 172.21.15.57:48490) 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.584+0000 7ff4ba800640 1 -- 172.21.15.57:0/1163562624 learned_addr learned my addr 172.21.15.57:0/1163562624 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4ba800640 1 -- 172.21.15.57:0/1163562624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc104480 msgr2=0x7ff4bc19a970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4ba800640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc104480 0x7ff4bc19a970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4ba800640 1 -- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 msgr2=0x7ff4bc19a430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4b9e00640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 0x7ff4bc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.586 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4ba800640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 0x7ff4bc19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.585+0000 7ff4ba800640 1 -- 172.21.15.57:0/1163562624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4bc19f420 con 0x7ff4bc109b80 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.586+0000 7ff4ba800640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400eaa0 tx=0x7ff4b400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.586+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400cdb0 con 0x7ff4bc109b80 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.587+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b4005020 con 0x7ff4bc109b80 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.587+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40116b0 con 0x7ff4bc109b80 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.587+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4bc19f710 con 0x7ff4bc109b80 2024-09-12T03:17:15.587 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.587+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4bc076630 con 0x7ff4bc109b80 2024-09-12T03:17:15.592 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.588+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4bc1051e0 con 0x7ff4bc109b80 2024-09-12T03:17:15.593 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.592+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ff4b40051d0 con 0x7ff4bc109b80 2024-09-12T03:17:15.595 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.595+0000 7ff4b2a00640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 0x7ff4a00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.595+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7ff4b40cfb40 con 0x7ff4bc109b80 2024-09-12T03:17:15.596 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.596+0000 7ff4b9e00640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 0x7ff4a00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.598 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.598+0000 7ff4b9e00640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 0x7ff4a00a5300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0096c0 tx=0x7ff4ac005fd0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.601+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4b4016030 con 0x7ff4bc109b80 2024-09-12T03:17:15.975 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.975+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4bc10c080 con 0x7ff4bc109b80 2024-09-12T03:17:15.977 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.977+0000 7ff4b2a00640 1 -- 172.21.15.57:0/1163562624 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25608 (secure 0 0 0) 0x7ff4b409c2a0 con 0x7ff4bc109b80 2024-09-12T03:17:15.980 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:17:15.980 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":108,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:17:06.873965+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2696288462":"2024-09-13T03:05:55.702194+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1354301616":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1679473944":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/2870728447":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6801/480086157":"2024-09-13T03:05:55.702194+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/480086157":"2024-09-13T03:05:55.702194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:15.983 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.983+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 msgr2=0x7ff4a00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.983+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 0x7ff4a00a5300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0096c0 tx=0x7ff4ac005fd0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.983+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 msgr2=0x7ff4bc19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.983+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff4b400eaa0 tx=0x7ff4b400ef60 comp rx=0 tx=0).stop 2024-09-12T03:17:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 shutdown_connections 2024-09-12T03:17:15.984 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4bc109b80 0x7ff4bc19ecf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4bc104480 0x7ff4bc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4a00a2e40 0x7ff4a00a5300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 --2- 172.21.15.57:0/1163562624 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4bc103ae0 0x7ff4bc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.984+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 >> 172.21.15.57:0/1163562624 conn(0x7ff4bc0ff7c0 msgr2=0x7ff4bc100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.985+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 shutdown_connections 2024-09-12T03:17:15.985 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:15.985+0000 7ff4bb200640 1 -- 172.21.15.57:0/1163562624 wait complete. 2024-09-12T03:17:16.013 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:58:06.250728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:03:03.359994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:03:06.244623+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-12T03:03:16.018628+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-12T03:17:16.014 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:17:16.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:15 smithi181 ceph-mon[35320]: pgmap v353: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:16.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:15 smithi181 ceph-mon[35320]: from='client.26110 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.314+0000 7f6ee7600640 1 -- 172.21.15.57:0/555020980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 msgr2=0x7f6ee0110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.314+0000 7f6ee7600640 1 --2- 172.21.15.57:0/555020980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0110730 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009ca0 tx=0x7f6ed802a970 comp rx=0 tx=0).stop 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 -- 172.21.15.57:0/555020980 shutdown_connections 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 --2- 172.21.15.57:0/555020980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0110730 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 --2- 172.21.15.57:0/555020980 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee0074ee0 0x7f6ee010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 --2- 172.21.15.57:0/555020980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee00745c0 0x7f6ee00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.315 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 -- 172.21.15.57:0/555020980 >> 172.21.15.57:0/555020980 conn(0x7f6ee00fd7c0 msgr2=0x7f6ee00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.315+0000 7f6ee7600640 1 -- 172.21.15.57:0/555020980 shutdown_connections 2024-09-12T03:17:16.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.316+0000 7f6ee7600640 1 -- 172.21.15.57:0/555020980 wait complete. 2024-09-12T03:17:16.317 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.317+0000 7f6ee7600640 1 Processor -- start 2024-09-12T03:17:16.319 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.318+0000 7f6ee7600640 1 -- start start 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 0x7f6ee0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0114120 con 0x7f6ee00745c0 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0113fa0 con 0x7f6ee010e320 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6ee7600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee01142a0 con 0x7f6ee0074ee0 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43348/0 (socket says 172.21.15.57:43348) 2024-09-12T03:17:16.320 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.319+0000 7f6edfe00640 1 -- 172.21.15.57:0/3998292158 learned_addr learned my addr 172.21.15.57:0/3998292158 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.320+0000 7f6ee4c00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 0x7f6ee0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.320+0000 7f6ee5600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.320+0000 7f6edfe00640 1 -- 172.21.15.57:0/3998292158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 msgr2=0x7f6ee0102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.320+0000 7f6edfe00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.320+0000 7f6edfe00640 1 -- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 msgr2=0x7f6ee0107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.321 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.321+0000 7f6edfe00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 0x7f6ee0107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.321+0000 7f6edfe00640 1 -- 172.21.15.57:0/3998292158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee0102e20 con 0x7f6ee0074ee0 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.321+0000 7f6ee4c00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 0x7f6ee0107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.321+0000 7f6edfe00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b4a0 tx=0x7f6ecc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.322+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc014070 con 0x7f6ee0074ee0 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.322+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc010af0 con 0x7f6ee0074ee0 2024-09-12T03:17:16.322 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.322+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee0103110 con 0x7f6ee0074ee0 2024-09-12T03:17:16.323 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.322+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc017d10 con 0x7f6ee0074ee0 2024-09-12T03:17:16.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.322+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee010d340 con 0x7f6ee0074ee0 2024-09-12T03:17:16.325 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.324+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0075c70 con 0x7f6ee0074ee0 2024-09-12T03:17:16.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.329+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f6ecc017070 con 0x7f6ee0074ee0 2024-09-12T03:17:16.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.331+0000 7f6edd600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 0x7f6ec80a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:16.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.331+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9642+0+0 (secure 0 0 0) 0x7f6ecc0ceb60 con 0x7f6ee0074ee0 2024-09-12T03:17:16.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.333+0000 7f6ee4c00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 0x7f6ec80a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:16.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.334+0000 7f6ee4c00640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 0x7f6ec80a5390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009b90 tx=0x7f6ed4005e50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:16.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.336+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ecc09b3d0 con 0x7f6ee0074ee0 2024-09-12T03:17:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:15 smithi057 ceph-mon[30361]: pgmap v353: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:16.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:15 smithi057 ceph-mon[30361]: from='client.26110 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:16.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:15 smithi204 ceph-mon[35013]: pgmap v353: 129 pgs: 5 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 120 active+clean; 10 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-09-12T03:17:16.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:15 smithi204 ceph-mon[35013]: from='client.26110 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:16.730 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.729+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ee010de20 con 0x7f6ee0074ee0 2024-09-12T03:17:16.731 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.730+0000 7f6edd600640 1 -- 172.21.15.57:0/3998292158 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f6ecc09acd0 con 0x7f6ee0074ee0 2024-09-12T03:17:16.733 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 1 2024-09-12T03:17:16.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 msgr2=0x7f6ec80a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.734 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 0x7f6ec80a5390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009b90 tx=0x7f6ed4005e50 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 msgr2=0x7f6ee0107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b4a0 tx=0x7f6ecc00b960 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 shutdown_connections 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ee010e320 0x7f6ee0102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee0074ee0 0x7f6ee0107990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6ec80a2ed0 0x7f6ec80a5390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 --2- 172.21.15.57:0/3998292158 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6ee00745c0 0x7f6ee0107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.734+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 >> 172.21.15.57:0/3998292158 conn(0x7f6ee00fd7c0 msgr2=0x7f6ee00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:16.735 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.735+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 shutdown_connections 2024-09-12T03:17:16.736 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:16.735+0000 7f6ee7600640 1 -- 172.21.15.57:0/3998292158 wait complete. 2024-09-12T03:17:16.763 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:17:17.050 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 -- 172.21.15.57:0/1719250355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904102420 msgr2=0x7f490410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 --2- 172.21.15.57:0/1719250355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904102420 0x7f490410e950 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009ca0 tx=0x7f48fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 -- 172.21.15.57:0/1719250355 shutdown_connections 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 --2- 172.21.15.57:0/1719250355 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904102420 0x7f490410e950 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 --2- 172.21.15.57:0/1719250355 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904101a80 0x7f4904101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 --2- 172.21.15.57:0/1719250355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904107a90 0x7f4904107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.051 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.050+0000 7f490a200640 1 -- 172.21.15.57:0/1719250355 >> 172.21.15.57:0/1719250355 conn(0x7f49040fd7c0 msgr2=0x7f49040ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.052+0000 7f490a200640 1 -- 172.21.15.57:0/1719250355 shutdown_connections 2024-09-12T03:17:17.054 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.052+0000 7f490a200640 1 -- 172.21.15.57:0/1719250355 wait complete. 2024-09-12T03:17:17.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.054+0000 7f490a200640 1 Processor -- start 2024-09-12T03:17:17.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.054+0000 7f490a200640 1 -- start start 2024-09-12T03:17:17.055 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.054+0000 7f490a200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 0x7f49041a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f490a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f490a200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904107a90 0x7f49041abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f490a200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f490410fd60 con 0x7f4904107a90 2024-09-12T03:17:17.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f490a200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f490410fbe0 con 0x7f4904101a80 2024-09-12T03:17:17.056 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f490a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f490410fee0 con 0x7f4904102420 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f4902a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f4903400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 0x7f49041a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f4902a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43362/0 (socket says 172.21.15.57:43362) 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f4902a00640 1 -- 172.21.15.57:0/1338424247 learned_addr learned my addr 172.21.15.57:0/1338424247 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.055+0000 7f4903e00640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904107a90 0x7f49041abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 -- 172.21.15.57:0/1338424247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 msgr2=0x7f49041a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 -- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904107a90 msgr2=0x7f49041abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904107a90 0x7f49041abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 -- 172.21.15.57:0/1338424247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49041ac1a0 con 0x7f4904101a80 2024-09-12T03:17:17.057 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4902a00640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:17.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.056+0000 7f4903400640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 0x7f49041a71e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009fd0 tx=0x7f48f400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.057+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4015020 con 0x7f4904101a80 2024-09-12T03:17:17.058 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.057+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49041ac490 con 0x7f4904101a80 2024-09-12T03:17:17.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.057+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490406bc10 con 0x7f4904101a80 2024-09-12T03:17:17.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.058+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f4011d10 con 0x7f4904101a80 2024-09-12T03:17:17.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.058+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f400f2e0 con 0x7f4904101a80 2024-09-12T03:17:17.062 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.058+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4904101ee0 con 0x7f4904101a80 2024-09-12T03:17:17.063 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.063+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f48f4018020 con 0x7f4904101a80 2024-09-12T03:17:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.066+0000 7f48fbe00640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 0x7f48e80a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.067 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.067+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9566+0+0 (secure 0 0 0) 0x7f48f40cf140 con 0x7f4904101a80 2024-09-12T03:17:17.070 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.068+0000 7f4902a00640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 0x7f48e80a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.071 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.067+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48f409b3b0 con 0x7f4904101a80 2024-09-12T03:17:17.072 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.072+0000 7f4902a00640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 0x7f48e80a52f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48f0011e00 tx=0x7f48f0013040 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:16 smithi057 ceph-mon[30361]: from='client.15942 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:16 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1163562624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:16 smithi057 ceph-mon[30361]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:17:17.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:16 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3998292158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:17:17.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:16 smithi181 ceph-mon[35320]: from='client.15942 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:17.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:16 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1163562624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:17.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:16 smithi181 ceph-mon[35320]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:17:17.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:16 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3998292158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:17:17.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.280+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f490406bf20 con 0x7f4904101a80 2024-09-12T03:17:17.281 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.281+0000 7f48fbe00640 1 -- 172.21.15.57:0/1338424247 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f48f409f6b0 con 0x7f4904101a80 2024-09-12T03:17:17.282 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 msgr2=0x7f48e80a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 0x7f48e80a52f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48f0011e00 tx=0x7f48f0013040 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 msgr2=0x7f49041a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 0x7f49041a71e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009fd0 tx=0x7f48f400b870 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 shutdown_connections 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f48e80a2e30 0x7f48e80a52f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f4904107a90 0x7f49041abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4904102420 0x7f49041a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 --2- 172.21.15.57:0/1338424247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4904101a80 0x7f49041a71e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 >> 172.21.15.57:0/1338424247 conn(0x7f49040fd7c0 msgr2=0x7f49040feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.284 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 shutdown_connections 2024-09-12T03:17:17.285 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.283+0000 7f490a200640 1 -- 172.21.15.57:0/1338424247 wait complete. 2024-09-12T03:17:17.299 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:17:17.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:16 smithi204 ceph-mon[35013]: from='client.15942 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:17.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1163562624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:17.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:16 smithi204 ceph-mon[35013]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T03:17:17.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:16 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3998292158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:17:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 -- 172.21.15.57:0/2257165748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 msgr2=0x7f05ec114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 --2- 172.21.15.57:0/2257165748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec114c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009ca0 tx=0x7f05e402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 -- 172.21.15.57:0/2257165748 shutdown_connections 2024-09-12T03:17:17.461 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 --2- 172.21.15.57:0/2257165748 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec114c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 --2- 172.21.15.57:0/2257165748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 0x7f05ec1122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 --2- 172.21.15.57:0/2257165748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 -- 172.21.15.57:0/2257165748 >> 172.21.15.57:0/2257165748 conn(0x7f05ec0ff840 msgr2=0x7f05ec101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 -- 172.21.15.57:0/2257165748 shutdown_connections 2024-09-12T03:17:17.462 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.460+0000 7f05eb200640 1 -- 172.21.15.57:0/2257165748 wait complete. 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.461+0000 7f05eb200640 1 Processor -- start 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 -- start start 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 0x7f05ec1a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec118370 con 0x7f05ec112820 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1181f0 con 0x7f05ec077040 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05eb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1184f0 con 0x7f05ec076720 2024-09-12T03:17:17.464 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:43366/0 (socket says 172.21.15.57:43366) 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 -- 172.21.15.57:0/376771766 learned_addr learned my addr 172.21.15.57:0/376771766 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05ea800640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e3e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 0x7f05ec1a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 -- 172.21.15.57:0/376771766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 msgr2=0x7f05ec1a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 0x7f05ec1a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 -- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 msgr2=0x7f05ec19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.462+0000 7f05e9e00640 1 -- 172.21.15.57:0/376771766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19ef60 con 0x7f05ec076720 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05ea800640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05e9e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9f0 tx=0x7f05dc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00cd80 con 0x7f05ec076720 2024-09-12T03:17:17.465 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc005500 con 0x7f05ec076720 2024-09-12T03:17:17.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010690 con 0x7f05ec076720 2024-09-12T03:17:17.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19f250 con 0x7f05ec076720 2024-09-12T03:17:17.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.463+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1ab560 con 0x7f05ec076720 2024-09-12T03:17:17.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.464+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec077e60 con 0x7f05ec076720 2024-09-12T03:17:17.466 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.466+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f05dc0046c0 con 0x7f05ec076720 2024-09-12T03:17:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.467+0000 7f05e2a00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 0x7f05d00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.467+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9566+0+0 (secure 0 0 0) 0x7f05dc0cff80 con 0x7f05ec076720 2024-09-12T03:17:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.468+0000 7f05e3e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 0x7f05d00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.468 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.468+0000 7f05e3e00640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 0x7f05d00a5240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05ec19fef0 tx=0x7f05d4005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.469 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.469+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05dc0541b0 con 0x7f05ec076720 2024-09-12T03:17:17.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.656+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f05ec19f590 con 0x7f05ec076720 2024-09-12T03:17:17.657 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.657+0000 7f05e2a00640 1 -- 172.21.15.57:0/376771766 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f05dc09c730 con 0x7f05ec076720 2024-09-12T03:17:17.658 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 64 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 msgr2=0x7f05d00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 0x7f05d00a5240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f05ec19fef0 tx=0x7f05d4005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 msgr2=0x7f05ec1a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9f0 tx=0x7f05dc00edf0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 shutdown_connections 2024-09-12T03:17:17.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f05d00a2d80 0x7f05d00a5240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f05ec112820 0x7f05ec19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ec077040 0x7f05ec1a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.659+0000 7f05eb200640 1 --2- 172.21.15.57:0/376771766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec076720 0x7f05ec1a45e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.660+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 >> 172.21.15.57:0/376771766 conn(0x7f05ec0ff840 msgr2=0x7f05ec113e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.660+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 shutdown_connections 2024-09-12T03:17:17.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.660+0000 7f05eb200640 1 -- 172.21.15.57:0/376771766 wait complete. 2024-09-12T03:17:17.676 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-12T03:17:17.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.838+0000 7fee36000640 1 -- 172.21.15.57:0/2881390762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30103df0 msgr2=0x7fee301041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.838+0000 7fee36000640 1 --2- 172.21.15.57:0/2881390762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30103df0 0x7fee301041d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee24009900 tx=0x7fee2402a410 comp rx=0 tx=0).stop 2024-09-12T03:17:17.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 -- 172.21.15.57:0/2881390762 shutdown_connections 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 --2- 172.21.15.57:0/2881390762 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee30108f40 0x7fee30110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 --2- 172.21.15.57:0/2881390762 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee301047a0 0x7fee301087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 --2- 172.21.15.57:0/2881390762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30103df0 0x7fee301041d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 -- 172.21.15.57:0/2881390762 >> 172.21.15.57:0/2881390762 conn(0x7fee300ff840 msgr2=0x7fee30101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 -- 172.21.15.57:0/2881390762 shutdown_connections 2024-09-12T03:17:17.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.839+0000 7fee36000640 1 -- 172.21.15.57:0/2881390762 wait complete. 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 Processor -- start 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 -- start start 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee30103df0 0x7fee3019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30108f40 0x7fee3019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fee30113ff0 con 0x7fee3019ed80 2024-09-12T03:17:17.842 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fee30113e70 con 0x7fee30103df0 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee36000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fee30114170 con 0x7fee30108f40 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee35600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee35600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:45846/0 (socket says 172.21.15.57:45846) 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee34c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee30103df0 0x7fee3019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee35600640 1 -- 172.21.15.57:0/1100215559 learned_addr learned my addr 172.21.15.57:0/1100215559 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.840+0000 7fee2fe00640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30108f40 0x7fee3019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 -- 172.21.15.57:0/1100215559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30108f40 msgr2=0x7fee3019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30108f40 0x7fee3019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 -- 172.21.15.57:0/1100215559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee30103df0 msgr2=0x7fee3019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee30103df0 0x7fee3019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 -- 172.21.15.57:0/1100215559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3019f870 con 0x7fee3019ed80 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee35600640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb10 tx=0x7fee2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28004b20 con 0x7fee3019ed80 2024-09-12T03:17:17.843 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3019fb60 con 0x7fee3019ed80 2024-09-12T03:17:17.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee28004cc0 con 0x7fee3019ed80 2024-09-12T03:17:17.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.841+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28010760 con 0x7fee3019ed80 2024-09-12T03:17:17.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.842+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a7300 con 0x7fee3019ed80 2024-09-12T03:17:17.844 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.842+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee301041d0 con 0x7fee3019ed80 2024-09-12T03:17:17.845 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.845+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fee28005000 con 0x7fee3019ed80 2024-09-12T03:17:17.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.846+0000 7fee2d600640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 0x7fee180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.846 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.846+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9566+0+0 (secure 0 0 0) 0x7fee280cffd0 con 0x7fee3019ed80 2024-09-12T03:17:17.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.846+0000 7fee34c00640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 0x7fee180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.847 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.847+0000 7fee34c00640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 0x7fee180a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee24000c00 tx=0x7fee24005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.848 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:17.848+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee280541b0 con 0x7fee3019ed80 2024-09-12T03:17:18.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.039+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fee3010c150 con 0x7fee3019ed80 2024-09-12T03:17:18.040 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.040+0000 7fee2d600640 1 -- 172.21.15.57:0/1100215559 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v109) ==== 103+0+11 (secure 0 0 0) 0x7fee2809c780 con 0x7fee3019ed80 2024-09-12T03:17:18.041 INFO:teuthology.orchestra.run.smithi057.stdout:pg_num: 32 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 msgr2=0x7fee180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 0x7fee180a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee24000c00 tx=0x7fee24005b50 comp rx=0 tx=0).stop 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 msgr2=0x7fee3019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee2800eb10 tx=0x7fee2800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 shutdown_connections 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fee180a2d00 0x7fee180a51c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fee3019ed80 0x7fee3019f230 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.043 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee30108f40 0x7fee3019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 --2- 172.21.15.57:0/1100215559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee30103df0 0x7fee3019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 >> 172.21.15.57:0/1100215559 conn(0x7fee300ff840 msgr2=0x7fee30100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 shutdown_connections 2024-09-12T03:17:18.044 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.042+0000 7fee36000640 1 -- 172.21.15.57:0/1100215559 wait complete. 2024-09-12T03:17:18.058 DEBUG:teuthology.orchestra.run.smithi057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:17:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:17 smithi181 ceph-mon[35320]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 79 op/s 2024-09-12T03:17:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:17 smithi181 ceph-mon[35320]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:17 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1338424247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:17 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/376771766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 -- 172.21.15.57:0/1648314950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c102420 msgr2=0x7fc55c10e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 --2- 172.21.15.57:0/1648314950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c102420 0x7fc55c10e950 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc554009cd0 tx=0x7fc55402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 -- 172.21.15.57:0/1648314950 shutdown_connections 2024-09-12T03:17:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 --2- 172.21.15.57:0/1648314950 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c102420 0x7fc55c10e950 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.218 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 --2- 172.21.15.57:0/1648314950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c101a80 0x7fc55c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 --2- 172.21.15.57:0/1648314950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c107a90 0x7fc55c107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 -- 172.21.15.57:0/1648314950 >> 172.21.15.57:0/1648314950 conn(0x7fc55c0fd7c0 msgr2=0x7fc55c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:18.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 -- 172.21.15.57:0/1648314950 shutdown_connections 2024-09-12T03:17:18.219 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.217+0000 7fc564600640 1 -- 172.21.15.57:0/1648314950 wait complete. 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.218+0000 7fc564600640 1 Processor -- start 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.218+0000 7fc564600640 1 -- start start 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 0x7fc55c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 0x7fc55c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c111f20 con 0x7fc55c107a90 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c111da0 con 0x7fc55c102420 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc564600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc55c1120a0 con 0x7fc55c101a80 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc561c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.221 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc562600640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 0x7fc55c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc561c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:33668/0 (socket says 172.21.15.57:33668) 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc561c00640 1 -- 172.21.15.57:0/214908169 learned_addr learned my addr 172.21.15.57:0/214908169 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.219+0000 7fc561200640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 0x7fc55c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 -- 172.21.15.57:0/214908169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 msgr2=0x7fc55c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 -- 172.21.15.57:0/214908169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 msgr2=0x7fc55c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 0x7fc55c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 -- 172.21.15.57:0/214908169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55c19f5a0 con 0x7fc55c107a90 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc561200640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 0x7fc55c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc561c00640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc562600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 0x7fc55c19ee70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc554009ca0 tx=0x7fc554031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc554004040 con 0x7fc55c107a90 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5540046c0 con 0x7fc55c107a90 2024-09-12T03:17:18.222 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55c19f830 con 0x7fc55c107a90 2024-09-12T03:17:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc554030e40 con 0x7fc55c107a90 2024-09-12T03:17:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.220+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55c1a7120 con 0x7fc55c107a90 2024-09-12T03:17:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.221+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55c101ee0 con 0x7fc55c107a90 2024-09-12T03:17:18.223 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.223+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fc5540305a0 con 0x7fc55c107a90 2024-09-12T03:17:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.225+0000 7fc55a000640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 0x7fc5440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.225+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9566+0+0 (secure 0 0 0) 0x7fc55402e030 con 0x7fc55c107a90 2024-09-12T03:17:18.225 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.225+0000 7fc561c00640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 0x7fc5440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:18.226 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.225+0000 7fc561c00640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 0x7fc5440a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc550011440 tx=0x7fc550005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:18.227 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.227+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc55404b060 con 0x7fc55c107a90 2024-09-12T03:17:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:17 smithi057 ceph-mon[30361]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 79 op/s 2024-09-12T03:17:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:17 smithi057 ceph-mon[30361]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:17 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1338424247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:17 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/376771766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:17 smithi204 ceph-mon[35013]: pgmap v354: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 79 op/s 2024-09-12T03:17:18.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:17 smithi204 ceph-mon[35013]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:18.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:17 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1338424247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:17 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/376771766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:17:18.463 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.463+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc55c1a7350 con 0x7fc55c107a90 2024-09-12T03:17:18.475 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.474+0000 7fc55a000640 1 -- 172.21.15.57:0/214908169 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+740898 (secure 0 0 0) 0x7fc5540bd8b0 con 0x7fc55c107a90 2024-09-12T03:17:18.485 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:17:18.485 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:17:18.485 INFO:teuthology.orchestra.run.smithi057.stdout: "epoch": 27, 2024-09-12T03:17:18.485 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.485 INFO:teuthology.orchestra.run.smithi057.stdout: "active_gid": 15126, 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "active_name": "x", 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addrs": { 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6800", 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 260348644 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v1", 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:6801", 2024-09-12T03:17:18.486 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 260348644 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "active_addr": "172.21.15.57:6801/260348644", 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "active_change": "2024-09-12T03:05:55.702433+0000", 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "available": true, 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "standbys": [ 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 25163, 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "y", 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:17:18.487 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.490 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.491 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.492 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.493 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.494 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:17:18.495 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.496 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.497 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.498 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.499 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.500 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.501 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.503 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.504 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:17:18.505 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.506 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:18.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-09-12T03:17:18.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.509 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-09-12T03:17:18.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-09-12T03:17:18.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.512 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.513 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.514 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.515 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.516 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.517 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-09-12T03:17:18.518 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:18.519 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-09-12T03:17:18.520 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.521 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:17:18.522 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.523 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.525 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.526 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:17:18.527 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.528 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.529 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.530 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.532 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.533 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.534 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.535 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.536 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.538 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:17:18.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-09-12T03:17:18.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.541 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.542 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:17:18.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.545 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.546 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.548 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.549 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-09-12T03:17:18.550 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-09-12T03:17:18.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.552 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.553 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.555 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:17:18.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:17:18.558 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:17:18.559 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.560 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.565 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.567 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.569 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.570 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.571 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.573 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.574 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.575 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.576 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-09-12T03:17:18.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.578 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.580 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-09-12T03:17:18.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-09-12T03:17:18.582 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-09-12T03:17:18.583 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-09-12T03:17:18.584 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.585 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.588 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-09-12T03:17:18.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.589 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.590 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.596 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.597 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.598 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.600 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.602 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.603 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.604 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.606 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.607 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.609 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.610 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.611 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.612 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.613 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:17:18.614 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:17:18.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.616 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.619 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.620 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.623 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.624 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.626 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.627 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.629 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.634 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.635 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.636 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:17:18.638 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:17:18.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.640 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.643 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.644 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.645 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.646 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.648 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.650 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.651 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.652 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.653 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.654 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.657 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.658 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.659 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.663 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-09-12T03:17:18.664 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:18.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.669 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.670 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.671 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.672 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.673 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.675 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.676 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.677 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:17:18.678 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:17:18.679 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.681 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.682 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.684 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.686 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.687 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.689 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.691 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.692 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.693 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:17:18.694 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.696 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.698 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.700 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.702 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.703 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.705 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.706 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.708 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-09-12T03:17:18.709 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.712 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.713 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.714 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.715 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-09-12T03:17:18.716 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.717 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.718 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.720 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.721 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.722 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.723 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.724 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.725 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.728 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.730 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.732 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.733 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.734 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.735 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.736 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-09-12T03:17:18.739 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.742 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.743 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.745 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.746 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.748 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.749 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.752 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.753 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.754 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.755 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.757 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.758 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.760 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.761 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.762 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.763 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.764 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:18.766 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.767 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.768 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.769 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.770 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.772 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.774 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.775 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.776 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.777 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.778 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.779 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.780 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.782 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.783 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.784 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.786 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.787 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.789 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.790 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.791 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.792 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.792 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.792 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.793 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.797 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:17:18.798 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.799 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-09-12T03:17:18.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.802 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.803 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.804 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.806 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.809 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.810 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.811 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.812 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.814 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.815 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.816 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.817 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.819 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.821 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.826 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.827 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.828 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.829 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.830 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.832 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.833 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.835 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.838 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.839 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.840 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.842 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.843 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.844 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.846 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.848 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.849 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.851 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.852 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-09-12T03:17:18.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-09-12T03:17:18.854 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-09-12T03:17:18.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-09-12T03:17:18.856 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.857 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.858 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.859 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.861 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.862 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.864 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.866 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.869 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.872 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.875 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.877 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.878 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.880 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.881 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.883 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.884 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.886 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-09-12T03:17:18.887 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.888 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.889 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.890 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.891 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.893 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.894 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.895 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.896 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.897 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.898 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.899 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-09-12T03:17:18.900 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-09-12T03:17:18.901 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.902 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.904 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.906 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.907 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.908 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.909 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.911 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.912 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.913 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.914 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.915 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-09-12T03:17:18.916 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.917 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.919 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.920 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.921 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.922 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.923 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.924 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.926 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:17:18.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.929 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.930 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.932 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.936 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.937 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.938 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.939 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.940 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.941 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.942 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.943 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "gid": 25174, 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "z", 2024-09-12T03:17:18.944 INFO:teuthology.orchestra.run.smithi057.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.945 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.946 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.947 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.948 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.949 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.950 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.951 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.952 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.953 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.954 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.955 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-09-12T03:17:18.956 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-09-12T03:17:18.957 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.958 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.959 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.960 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.961 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.962 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.963 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.964 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-09-12T03:17:18.965 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-09-12T03:17:18.966 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-09-12T03:17:18.967 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:18.968 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.969 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.970 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.971 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.972 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.973 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.974 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-09-12T03:17:18.975 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.976 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.977 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-09-12T03:17:18.978 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-09-12T03:17:18.979 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.980 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.981 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.982 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.983 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.984 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-09-12T03:17:18.985 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:17:18.986 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-09-12T03:17:18.987 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.988 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:17:18.989 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.990 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.991 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.992 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.993 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.994 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.995 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.996 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:17:18.997 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:18.998 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:18.999 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.000 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-09-12T03:17:19.001 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-09-12T03:17:19.002 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-09-12T03:17:19.003 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-09-12T03:17:19.004 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-09-12T03:17:19.005 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.006 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.007 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.008 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.009 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.010 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.011 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:17:19.012 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:17:19.013 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.014 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.015 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.016 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.017 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.018 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.019 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.020 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.021 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.022 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.023 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.024 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.025 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.026 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.027 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.028 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.029 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-09-12T03:17:19.030 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:19.031 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.032 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.033 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.034 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.035 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.036 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.037 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.038 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.039 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:17:19.040 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:17:19.041 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:17:19.042 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:17:19.043 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.044 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.045 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.046 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.047 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.048 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.049 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.050 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.051 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.052 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.053 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.054 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.055 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.056 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:17:19.057 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:17:19.058 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.059 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.060 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.061 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.062 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.063 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.064 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.065 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.066 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.067 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.068 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.069 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.070 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.071 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:17:19.072 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:17:19.073 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:17:19.074 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:17:19.075 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:17:19.076 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:17:19.077 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.078 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.079 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.082 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.083 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.084 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.085 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.086 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.087 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.088 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.089 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.090 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.091 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.092 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.093 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.094 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.095 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.096 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.097 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.098 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.099 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.100 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.101 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.102 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.103 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.104 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.105 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.106 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.107 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.108 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.109 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.110 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-09-12T03:17:19.111 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.112 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.113 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.114 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.115 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.116 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.117 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.118 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.119 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-09-12T03:17:19.120 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.121 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.122 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.123 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.124 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.125 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.126 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.127 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.128 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.129 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.130 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.131 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.132 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.133 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.134 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.135 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.136 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.137 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.138 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.139 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.140 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-09-12T03:17:19.141 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.142 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-09-12T03:17:19.143 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.144 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:17:19.145 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.146 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:17:19.147 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.148 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.149 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.150 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.151 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.152 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.153 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.154 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.155 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.156 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.157 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-09-12T03:17:19.158 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.159 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.160 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.161 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.162 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.163 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.164 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.165 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.166 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.167 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.168 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.169 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.170 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.171 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.172 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.173 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.174 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.175 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.176 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.177 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.178 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.179 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.180 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.181 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.182 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.183 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.184 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.185 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.186 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.187 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.188 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.189 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.190 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.191 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.192 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.193 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.194 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.195 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.196 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.197 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.200 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.200 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.200 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.200 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.200 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.201 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.202 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.203 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.204 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.205 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.206 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.207 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.208 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.209 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.210 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.211 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.212 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.213 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.214 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.215 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.216 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.217 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.218 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-09-12T03:17:19.219 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.220 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.221 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.222 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.223 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.224 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.225 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.226 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.227 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.228 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.229 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.230 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.231 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.232 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.233 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.234 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.235 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.236 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.237 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.238 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.239 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.240 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.241 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.242 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.243 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.244 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.245 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.246 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-09-12T03:17:19.247 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.248 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.249 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.250 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:17:19.251 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.252 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.253 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.254 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.255 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.256 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.257 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.258 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.259 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.260 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.261 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.262 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-09-12T03:17:19.263 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:19.264 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.265 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.266 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.267 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.268 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.269 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.270 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.271 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.272 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.273 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.274 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.275 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.276 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.277 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.278 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.279 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.280 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.281 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.282 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.283 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.284 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.285 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.286 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.287 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.288 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.289 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.290 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.291 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.292 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.293 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.294 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.295 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.296 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.297 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.298 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.299 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.300 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.301 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-09-12T03:17:19.302 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-09-12T03:17:19.303 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-09-12T03:17:19.304 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-09-12T03:17:19.305 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-09-12T03:17:19.306 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.307 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.308 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.309 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.310 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.311 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.312 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.313 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-09-12T03:17:19.314 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.315 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-09-12T03:17:19.316 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.317 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.318 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.319 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.320 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.321 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.322 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.323 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.324 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.325 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.326 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.327 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.328 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.329 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.330 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.331 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.332 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.333 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.334 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.335 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.336 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.337 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.338 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.339 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.340 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.341 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.342 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.343 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.344 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.345 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.346 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.347 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.348 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.349 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.350 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.351 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.352 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-09-12T03:17:19.353 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-09-12T03:17:19.354 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.355 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.356 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.357 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.358 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.359 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-09-12T03:17:19.360 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.361 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.362 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.363 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.364 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.365 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.366 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.367 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.368 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.369 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.370 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-09-12T03:17:19.371 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.372 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.373 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.374 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.375 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.376 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.377 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.378 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.379 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.380 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.381 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.382 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.383 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.384 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.385 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.386 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-09-12T03:17:19.387 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.388 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.389 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.390 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.391 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.392 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.393 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.394 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.395 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.396 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.397 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.398 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.399 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.400 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.401 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.402 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.403 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.404 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:17:19.405 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.408 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.408 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "modules": [ 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "iostat", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "nfs", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "restful", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "snap_schedule" 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "available_modules": [ 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "alerts", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.415 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.416 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.417 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_destination": { 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_destination", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.418 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_from_name": { 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_from_name", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "Ceph", 2024-09-12T03:17:19.419 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Email From: name", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_host": { 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_host", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.420 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP server", 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_password": { 2024-09-12T03:17:19.421 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_password", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Password to authenticate with", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.422 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_port": { 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_port", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "465", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP port", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.423 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_sender": { 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_sender", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.424 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_ssl": { 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_ssl", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.425 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "smtp_user": { 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "smtp_user", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.426 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "User to authenticate as", 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.427 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "balancer", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "active": { 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "active", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.428 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_time": { 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_time", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.429 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0000", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "begin_weekday": { 2024-09-12T03:17:19.430 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "begin_weekday", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:19.431 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_max_iterations": { 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "25", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "250", 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.432 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_metrics": { 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_metrics", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:17:19.433 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "crush_compat_step": { 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crush_compat_step", 2024-09-12T03:17:19.434 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.5", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0.001", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "0.999", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.435 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "end_time": { 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_time", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2359", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:17:19.436 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "end_weekday": { 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "end_weekday", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "0", 2024-09-12T03:17:19.437 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "6", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.438 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.439 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.440 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.441 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.442 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.443 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "min_score": { 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_score", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.444 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:19.445 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "upmap", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "crush-compat", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "none", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "read", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap", 2024-09-12T03:17:19.446 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap-read" 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Balancer mode", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_ids": { 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_ids", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.447 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.448 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.449 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_deviation": { 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_deviation", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.450 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "upmap_max_optimizations": { 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.451 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.452 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_down_multiplier": { 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_down_multiplier", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.453 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_refresh_rate": { 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_refresh_rate", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.454 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "20", 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.455 INFO:teuthology.orchestra.run.smithi057.stdout: "agent_starting_port": { 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "agent_starting_port", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4721", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.456 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_ptrace": { 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_ptrace", 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.457 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_interval": { 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_interval", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.458 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:17:19.459 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0.7", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.460 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm_log_destination": { 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cephadm_log_destination", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "file", 2024-09-12T03:17:19.461 INFO:teuthology.orchestra.run.smithi057.stdout: "file,syslog", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "syslog" 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "cgroups_split": { 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cgroups_split", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.462 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "config_checks_enabled": { 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_checks_enabled", 2024-09-12T03:17:19.463 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.464 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "config_dashboard": { 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_dashboard", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.465 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_alertmanager": { 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_alertmanager", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.466 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_base": { 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_base", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.467 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_elasticsearch": { 2024-09-12T03:17:19.468 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "elasticsearch container image", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.469 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_grafana": { 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_grafana", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.470 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_haproxy": { 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_haproxy", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:17:19.471 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "HAproxy container image", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_agent": { 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.472 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.473 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_collector": { 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.474 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_jaeger_query": { 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.475 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Jaeger query container image", 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_keepalived": { 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_keepalived", 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.476 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Keepalived container image", 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_loki": { 2024-09-12T03:17:19.477 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_loki", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Loki container image", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.478 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_node_exporter": { 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_node_exporter", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.479 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_nvmeof": { 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_nvmeof", 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:17:19.480 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Nvme-of container image", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_prometheus": { 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_prometheus", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.481 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Prometheus container image", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.482 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_promtail": { 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_promtail", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Promtail container image", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.483 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_samba": { 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_samba", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.484 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "container_image_snmp_gateway": { 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:17:19.485 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "container_init": { 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "container_init", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.486 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "daemon_cache_timeout": { 2024-09-12T03:17:19.487 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.488 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "900", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.489 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "default_registry": { 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "default_registry", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "docker.io", 2024-09-12T03:17:19.490 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "device_cache_timeout": { 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_cache_timeout", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.491 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1800", 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "device_enhanced_scan": { 2024-09-12T03:17:19.492 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_enhanced_scan", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.493 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "facts_cache_timeout": { 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "facts_cache_timeout", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.494 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "grafana_dashboards_path": { 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.495 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "host_check_interval": { 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "host_check_interval", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.496 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "hw_monitoring": { 2024-09-12T03:17:19.497 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hw_monitoring", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.498 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "inventory_list_all": { 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "inventory_list_all", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.499 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.500 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.501 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "log_refresh_metadata": { 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_refresh_metadata", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.502 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.503 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.504 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.505 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.506 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.507 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.508 INFO:teuthology.orchestra.run.smithi057.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "*", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.509 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max_count_per_host": { 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_count_per_host", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.510 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "max_osd_draining_count": { 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_osd_draining_count", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.511 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "migration_current": { 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "migration_current", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.512 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "internal - do not modify", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "mode": { 2024-09-12T03:17:19.513 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mode", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "root", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm-package", 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "root" 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.514 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "oob_default_addr": { 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "oob_default_addr", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "169.254.1.1", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.515 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "prometheus_alerts_path": { 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.516 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_insecure": { 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_insecure", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.517 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.518 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_password": { 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_password", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.519 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_url": { 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_url", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.520 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "registry_username": { 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "registry_username", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:17:19.521 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "secure_monitoring_stack": { 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.522 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "service_discovery_port": { 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "service_discovery_port", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.523 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8765", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "ssh_config_file": { 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssh_config_file", 2024-09-12T03:17:19.524 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.525 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "use_agent": { 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_agent", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:17:19.526 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "use_repo_digest": { 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "use_repo_digest", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.527 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_failed_host_check": { 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.528 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_daemons": { 2024-09-12T03:17:19.529 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.530 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_on_stray_hosts": { 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.531 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crash", 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.532 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.533 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.534 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.535 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.536 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.537 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.538 INFO:teuthology.orchestra.run.smithi057.stdout: "retain_interval": { 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retain_interval", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "31536000", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.539 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_recent_interval": { 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_recent_interval", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1209600", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:17:19.540 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dashboard", 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:17:19.541 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.542 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.543 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.544 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.545 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:17:19.546 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.547 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.548 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.549 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.550 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:17:19.551 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.552 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.553 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.554 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.555 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:17:19.556 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.557 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.558 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.559 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.560 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_URL": { 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.561 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:17:19.562 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.564 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.565 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.566 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:17:19.567 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.568 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.569 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.570 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.571 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:17:19.572 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.573 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.574 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.575 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.576 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:17:19.577 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.578 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.579 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.580 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:17:19.581 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.582 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.583 INFO:teuthology.orchestra.run.smithi057.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.584 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "45", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.585 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.586 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.587 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "admin", 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:17:19.588 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.589 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.590 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.591 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.592 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:17:19.593 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.594 INFO:teuthology.orchestra.run.smithi057.stdout: "cross_origin_url": { 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cross_origin_url", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.595 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "crt_file": { 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "crt_file", 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.596 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "debug": { 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "debug", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.597 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.598 INFO:teuthology.orchestra.run.smithi057.stdout: "jwt_token_ttl": { 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "jwt_token_ttl", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "28800", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.599 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.600 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.601 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.602 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.603 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.604 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.605 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.606 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "motd": { 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "motd", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.607 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "The message of the day", 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.608 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.609 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.613 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.614 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8080", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.615 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.616 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl_server_port": { 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl_server_port", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.617 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8443", 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:19.618 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "redirect", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "redirect" 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.619 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:19.620 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "url_prefix": { 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url_prefix", 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.622 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_monitoring": { 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_monitoring", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.623 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "monitor device health metrics", 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.624 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.625 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.626 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.627 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.628 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.629 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "mark_out_threshold": { 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mark_out_threshold", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.630 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "2419200", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "pool_name": { 2024-09-12T03:17:19.631 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pool_name", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "device_health_metrics", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.632 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "retention_period": { 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "retention_period", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15552000", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:17:19.633 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_frequency": { 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_frequency", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.634 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "self_heal": { 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "self_heal", 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.635 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.636 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.637 INFO:teuthology.orchestra.run.smithi057.stdout: "warn_threshold": { 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "warn_threshold", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7257600", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:17:19.638 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "diskprediction_local", 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.639 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.640 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.641 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.642 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.643 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.644 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.645 INFO:teuthology.orchestra.run.smithi057.stdout: "predict_interval": { 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predict_interval", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "86400", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.646 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "predictor_model": { 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "predictor_model", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "prophetstor", 2024-09-12T03:17:19.647 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.648 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "600", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.649 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "influx", 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": false, 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "batch_size": { 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "batch_size", 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.650 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5000", 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "database": { 2024-09-12T03:17:19.651 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "database", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "ceph", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.652 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "hostname": { 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "hostname", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:17:19.653 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "30", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "5", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.654 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.655 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.656 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.657 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.658 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.659 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.660 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "password": { 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "password", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.661 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "port": { 2024-09-12T03:17:19.662 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "port", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "8086", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "InfluxDB server port", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.663 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "ssl": { 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ssl", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "false", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:19.664 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "threads": { 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threads", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "32", 2024-09-12T03:17:19.665 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "username": { 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "username", 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.666 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "verify_ssl": { 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "verify_ssl", 2024-09-12T03:17:19.667 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "true", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.668 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "insights", 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.669 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.670 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.671 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.672 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.673 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.674 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "iostat", 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.675 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.676 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.677 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.678 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.679 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.680 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.681 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "k8sevents", 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_event_retention_days": { 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.682 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "7", 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "config_check_secs": { 2024-09-12T03:17:19.683 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "config_check_secs", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "10", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.684 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.685 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.686 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.687 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.688 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.689 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.690 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "localpool", 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "failure_domain": { 2024-09-12T03:17:19.691 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "failure_domain", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "host", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.692 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.693 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.694 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.695 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.696 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.697 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.698 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min_size": { 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "min_size", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.699 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "num_rep": { 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "num_rep", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.700 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_num": { 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_num", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.701 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "128", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "prefix": { 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prefix", 2024-09-12T03:17:19.702 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.703 INFO:teuthology.orchestra.run.smithi057.stdout: "subtree": { 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "subtree", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "rack", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.704 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mds_autoscaler", 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.705 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.706 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.707 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.708 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.709 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.710 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.711 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirroring", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.712 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.713 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.714 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.715 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.716 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.717 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "nfs", 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.718 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.719 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.720 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.721 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.722 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.723 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.724 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "fail_fs": { 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "fail_fs", 2024-09-12T03:17:19.725 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.726 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.727 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.728 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.729 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.730 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.731 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator": { 2024-09-12T03:17:19.732 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "orchestrator", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "cephadm", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "rook", 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "test_orchestrator" 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.733 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Orchestrator backend", 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_perf_query", 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.734 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.735 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.736 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.737 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.738 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.739 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.740 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "osd_support", 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.741 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.742 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.743 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.744 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.745 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.746 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.747 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "pg_autoscaler", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.748 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.749 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.750 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.751 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.752 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.753 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.754 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.755 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "threshold": { 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "threshold", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "3.0", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1.0", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.756 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "scaling threshold", 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "progress", 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.757 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_pg_recovery_event": { 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:17:19.758 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.759 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.760 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.761 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.762 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.763 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.764 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.765 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "max_completed_events": { 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_completed_events", 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.766 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "50", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "sleep_interval": { 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "sleep_interval", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.767 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "5", 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.768 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "prometheus", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "cache": { 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "cache", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.769 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "exclude_perf_counters": { 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "exclude_perf_counters", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.770 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.771 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.772 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.773 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.774 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.775 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.776 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools": { 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools", 2024-09-12T03:17:19.777 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.778 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "300", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.779 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "scrape_interval": { 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "scrape_interval", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15.0", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.780 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "::", 2024-09-12T03:17:19.781 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.782 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "9283", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "stale_cache_strategy": { 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stale_cache_strategy", 2024-09-12T03:17:19.783 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "log", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.784 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_behaviour": { 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_behaviour", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "default", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "default", 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: "error" 2024-09-12T03:17:19.785 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "standby_error_status_code": { 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "standby_error_status_code", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "500", 2024-09-12T03:17:19.786 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "400", 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "599", 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.787 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.788 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.789 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.790 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.791 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.792 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.793 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_snap_create": { 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.794 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.795 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "trash_purge_schedule": { 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "trash_purge_schedule", 2024-09-12T03:17:19.796 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.797 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "restful", 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "enable_auth": { 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enable_auth", 2024-09-12T03:17:19.798 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.799 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "key_file": { 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "key_file", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.800 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.801 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.802 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.803 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.804 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.805 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.806 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "server_addr": { 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_addr", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "server_port": { 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "server_port", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.808 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rgw", 2024-09-12T03:17:19.809 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.812 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.813 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.814 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.815 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rook", 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.816 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.817 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.818 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.819 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.820 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.821 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.822 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "storage_class": { 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "storage_class", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "local", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.823 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "selftest", 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.824 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.825 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.826 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.827 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.828 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.829 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "roption1": { 2024-09-12T03:17:19.830 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption1", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.831 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "roption2": { 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "roption2", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "xyz", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.832 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption1": { 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption1", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.833 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption2": { 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption2", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.834 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption3": { 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption3", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "float", 2024-09-12T03:17:19.835 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.836 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption4": { 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption4", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.837 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption5": { 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption5", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.838 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption6": { 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption6", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.839 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "rwoption7": { 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rwoption7", 2024-09-12T03:17:19.840 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "1", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "42", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.841 INFO:teuthology.orchestra.run.smithi057.stdout: "testkey": { 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testkey", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.842 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "testlkey": { 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testlkey", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.843 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "testnewline": { 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "testnewline", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.844 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.845 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "allow_m_granularity": { 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "allow_m_granularity", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:17:19.846 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "dump_on_update": { 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "dump_on_update", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.848 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.849 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.850 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.851 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.852 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.853 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.854 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "stats", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.855 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.856 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.857 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.858 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.859 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.860 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.861 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "status", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.862 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.863 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.864 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.865 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.866 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.867 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telegraf", 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "address": { 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "address", 2024-09-12T03:17:19.868 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.869 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "15", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.870 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.871 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.872 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.873 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.874 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.875 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.876 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "telemetry", 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.877 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_basic": { 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_basic", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:17:19.878 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_crash": { 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_crash", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.879 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_device": { 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_device", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.880 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_ident": { 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_ident", 2024-09-12T03:17:19.881 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.882 INFO:teuthology.orchestra.run.smithi057.stdout: "channel_perf": { 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "channel_perf", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.883 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "contact": { 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "contact", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.884 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "description": { 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "description", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.885 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "device_url": { 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "device_url", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.886 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "enabled": { 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "enabled", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.887 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.888 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "24", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "8", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.889 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "last_opt_revision": { 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "last_opt_revision", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "1", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.890 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard": { 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.891 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "leaderboard_description": { 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "leaderboard_description", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.892 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.893 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.894 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.895 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.896 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.897 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.898 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "organization": { 2024-09-12T03:17:19.899 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "organization", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.900 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "proxy": { 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "proxy", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.901 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "url": { 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "url", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:17:19.902 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "test_orchestrator", 2024-09-12T03:17:19.903 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.904 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.905 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.906 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.907 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.908 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.909 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.910 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.911 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.912 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.913 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.914 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.915 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "max_concurrent_clones": { 2024-09-12T03:17:19.916 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "max_concurrent_clones", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "4", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.917 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "periodic_async_work": { 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "periodic_async_work", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Periodically check for async work", 2024-09-12T03:17:19.918 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_delay": { 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "0", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.919 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.920 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "True", 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.921 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "can_run": true, 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "error_string": "", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "module_options": { 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "discovery_interval": { 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "discovery_interval", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "uint", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "100", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.922 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "identifier": { 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "identifier", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.923 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "interval": { 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "interval", 2024-09-12T03:17:19.924 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "secs", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "60", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.925 INFO:teuthology.orchestra.run.smithi057.stdout: "log_level": { 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_level", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.926 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster": { 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.927 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_cluster_level": { 2024-09-12T03:17:19.928 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_cluster_level", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "info", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [ 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "critical", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "debug", 2024-09-12T03:17:19.929 INFO:teuthology.orchestra.run.smithi057.stdout: "error", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "info", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "warning" 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "log_to_file": { 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "log_to_file", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "bool", 2024-09-12T03:17:19.930 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 1, 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "False", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_host": { 2024-09-12T03:17:19.931 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_host", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.932 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_port": { 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_port", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "int", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "10051", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.933 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "zabbix_sender": { 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "zabbix_sender", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "str", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "level": "advanced", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "flags": 0, 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:17:19.934 INFO:teuthology.orchestra.run.smithi057.stdout: "min": "", 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "max": "", 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "enum_allowed": [], 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "desc": "", 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "long_desc": "", 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "tags": [], 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "see_also": [] 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "services": { 2024-09-12T03:17:19.935 INFO:teuthology.orchestra.run.smithi057.stdout: "dashboard": "https://172.21.15.57:8443/" 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "always_on_modules": { 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "octopus": [ 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-09-12T03:17:19.936 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "pacific": [ 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-09-12T03:17:19.937 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "quincy": [ 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-09-12T03:17:19.938 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "reef": [ 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-09-12T03:17:19.939 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: ], 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "squid": [ 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "balancer", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "crash", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "devicehealth", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "orchestrator", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "pg_autoscaler", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "progress", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "rbd_support", 2024-09-12T03:17:19.940 INFO:teuthology.orchestra.run.smithi057.stdout: "status", 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "telemetry", 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "volumes" 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "last_failure_osd_epoch": 97, 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "active_clients": [ 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "devicehealth", 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.941 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 651199836 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "libcephsqlite", 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.942 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2751418299 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "rbd_support", 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 1685546426 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.943 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 640601239 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.944 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "snap_schedule", 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 3775908268 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.945 INFO:teuthology.orchestra.run.smithi057.stdout: "name": "volumes", 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: "addrvec": [ 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: { 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: "type": "v2", 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: "addr": "172.21.15.57:0", 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: "nonce": 2652407061 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout: ] 2024-09-12T03:17:19.946 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:17:19.948 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:18 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1100215559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:17:19.948 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:18 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/214908169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:19.948 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:18 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1100215559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:17:19.948 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:18 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/214908169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.481+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 msgr2=0x7fc5440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.481+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 0x7fc5440a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc550011440 tx=0x7fc550005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.481+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 msgr2=0x7fc55c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.481+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 0x7fc55c19ee70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc554009ca0 tx=0x7fc554031e60 comp rx=0 tx=0).stop 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 shutdown_connections 2024-09-12T03:17:19.949 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fc5440a2d00 0x7fc5440a51c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fc55c107a90 0x7fc55c19ee70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc55c102420 0x7fc55c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 --2- 172.21.15.57:0/214908169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc55c101a80 0x7fc55c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 >> 172.21.15.57:0/214908169 conn(0x7fc55c0fd7c0 msgr2=0x7fc55c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 shutdown_connections 2024-09-12T03:17:19.950 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:18.482+0000 7fc564600640 1 -- 172.21.15.57:0/214908169 wait complete. 2024-09-12T03:17:19.956 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:17:19.957 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:17:19.957 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:18 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1100215559' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-12T03:17:19.957 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:18 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/214908169' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:20.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:19 smithi181 ceph-mon[35320]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 229 op/s 2024-09-12T03:17:20.344 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:20.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:19 smithi057 ceph-mon[30361]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 229 op/s 2024-09-12T03:17:20.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:19 smithi204 ceph-mon[35013]: pgmap v356: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 229 op/s 2024-09-12T03:17:21.060 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 64, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 45, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 39057, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 44665, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 12, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.020653144, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.002552440, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.014660864, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000374050, 2024-09-12T03:17:21.061 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 44, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.048698810, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.001106791 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 68, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000010182, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000149 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 4000, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 1664 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.062 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 521, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 799, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 168481, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 848821, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 10, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 6, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.128656577, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.061068875, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.062579204, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.008027421, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 796, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.193771629, 2024-09-12T03:17:21.063 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000243431 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1080, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000322283, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000298 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 3616, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 478944 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 332, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 344, 2024-09-12T03:17:21.064 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1803303, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 195560, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 10, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 4, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.102284948, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.042968651, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.073211010, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.001027996, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 343, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.067111759, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000195661 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.065 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 54, 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000045245, 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000837 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1797200, 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 152816 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-09-12T03:17:21.066 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 1, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-MDSRank": { 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 34, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.002164870, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000063672 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:21.067 INFO:teuthology.orchestra.run.smithi057.stdout: "queue_len": 0, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "complete_latency": { 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 2, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000022703, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000011351 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "mds": { 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "request": 9, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "reply": 5, 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "reply_latency": { 2024-09-12T03:17:21.068 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 5, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.001141344, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000228268 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_reply": 0, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "forward": 4, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_commit": 11, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_split": 0, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_merge": 0, 2024-09-12T03:17:21.069 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes": 17, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_top": 1, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_bottom": 16, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pin_tail": 0, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_pinned": 13, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_expired": 8, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "inodes_with_caps": 1, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "caps": 2, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "subtrees": 4, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse": 21, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_hit": 13, 2024-09-12T03:17:21.070 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_forward": 1, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_discover": 5, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "traverse_lock": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "load_cent": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "q": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "exported": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "exported_inodes": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "imported": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "imported_inodes": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:21.071 INFO:teuthology.orchestra.run.smithi057.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rfiles": 4, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rbytes": 234, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "root_rsnaps": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:21.072 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:17:21.073 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_cap_release": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "process_request_cap_release": 1 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_cache": { 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_update_receipt": 12, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_try_discover": 8, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_send_discover": 10, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "dir_handle_discover": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_created": 0, 2024-09-12T03:17:21.074 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_enqueued": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "strays_migrated": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_started": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "recovery_completed": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_exportdir": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_flush": 0, 2024-09-12T03:17:21.075 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "ireq_inodestats": 0 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_log": { 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "evlrg": 0, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "evadd": 8, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "evex": 0, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "evtrm": 0, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "ev": 8, 2024-09-12T03:17:21.076 INFO:teuthology.orchestra.run.smithi057.stdout: "evexg": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "evexd": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segadd": 2, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segex": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segtrm": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "seg": 2, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segmjr": 2, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segexg": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "segexd": 0, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "expos": 4194304, 2024-09-12T03:17:21.077 INFO:teuthology.orchestra.run.smithi057.stdout: "wrpos": 4196664, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "rdpos": 0, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "jlat": { 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 7, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.024963609, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.003566229 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "replayed": 0 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_mem": { 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "ino": 22, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "ino+": 26, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "ino-": 4, 2024-09-12T03:17:21.078 INFO:teuthology.orchestra.run.smithi057.stdout: "dir": 21, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dir+": 26, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dir-": 5, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dn": 17, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dn+": 25, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dn-": 8, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "cap": 2, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "cap+": 3, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "cap-": 1, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "rss": 49020, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "heap": 223520 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_server": { 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_client_request": 10, 2024-09-12T03:17:21.079 INFO:teuthology.orchestra.run.smithi057.stdout: "dispatch_server_request": 1, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_request": 9, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_client_session": 26, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "handle_peer_request": 2, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "req_create_latency": { 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getattr_latency": { 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 4, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000972914, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000243228 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.080 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "req_link_latency": { 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookup_latency": { 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.081 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupino_latency": { 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 1, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000168430, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000168430 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupname_latency": { 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.082 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "req_lssnap_latency": { 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.083 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mkdir_latency": { 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mknod_latency": { 2024-09-12T03:17:21.084 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "req_mksnap_latency": { 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.085 INFO:teuthology.orchestra.run.smithi057.stdout: "req_open_latency": { 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "req_readdir_latency": { 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rename_latency": { 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.086 INFO:teuthology.orchestra.run.smithi057.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmdir_latency": { 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.087 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setattr_latency": { 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.088 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setlayout_latency": { 2024-09-12T03:17:21.089 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "req_setxattr_latency": { 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "req_symlink_latency": { 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.090 INFO:teuthology.orchestra.run.smithi057.stdout: "req_unlink_latency": { 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_sessions": { 2024-09-12T03:17:21.091 INFO:teuthology.orchestra.run.smithi057.stdout: "session_count": 2, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "session_add": 4, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "session_remove": 2, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_open": 2, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "sessions_stale": 0, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "total_load": 4, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "average_load": 1, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "avg_session_uptime": 682, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:21.092 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:21.093 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:21.094 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 271333, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 91, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:21.095 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 9904, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 200, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-09-12T03:17:21.096 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 92605, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 355, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "objecter": { 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_active": 0, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_laggy": 0, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send": 44, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_send_bytes": 12752, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_resend": 32, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_reply": 44, 2024-09-12T03:17:21.097 INFO:teuthology.orchestra.run.smithi057.stdout: "op_latency": { 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 44, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.308995006, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.007022613 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "op_inflight": 0, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "oplen_avg": { 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 44, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 66 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "op": 44, 2024-09-12T03:17:21.098 INFO:teuthology.orchestra.run.smithi057.stdout: "op_r": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "op_w": 44, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "op_rmw": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "op_pg": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_stat": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_create": 10, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_read": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_write": 7, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writefull": 8, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_writesame": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_append": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_zero": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_truncate": 0, 2024-09-12T03:17:21.099 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_delete": 6, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_mapext": 0, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_getxattr": 0, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_setxattr": 20, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:21.100 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_call": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_watch": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_notify": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "osdop_other": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_active": 0, 2024-09-12T03:17:21.101 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_send": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_resend": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "linger_ping": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_active": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_send": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolop_resend": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_active": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_send": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "poolstat_resend": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_active": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_send": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "statfs_resend": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "command_active": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "command_send": 0, 2024-09-12T03:17:21.102 INFO:teuthology.orchestra.run.smithi057.stdout: "command_resend": 0, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "map_epoch": 109, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "map_full": 0, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "map_inc": 15, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_sessions": 10, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_open": 10, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_session_close": 0, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_laggy": 0, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_wr": 14, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_rd": 0, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_del": 1 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "oft": { 2024-09-12T03:17:21.103 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_objs": 1, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_updates": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "omap_total_removes": 0 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "osd-slow-ops": { 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "slow_ops_count": 0 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "purge_queue": { 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executing_high_water": 0, 2024-09-12T03:17:21.104 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed_ops": 0, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_executed": 0, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 913, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1909375, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:21.105 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 913, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 913, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1909375, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-09-12T03:17:21.106 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 44, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 12752, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 44, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 12752, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.107 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 1024, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 44, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 44, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 44, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 44, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.108 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:21.109 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-write_buf_throttle-0x558d4a462960": { 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 3758096384, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 8, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 2360, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:21.110 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 8, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 7, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 2360, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:21.111 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:17:22.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:21 smithi181 ceph-mon[35320]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 243 op/s 2024-09-12T03:17:22.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:21 smithi181 ceph-mon[35320]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:17:22.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:21 smithi057 ceph-mon[30361]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 243 op/s 2024-09-12T03:17:22.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:21 smithi057 ceph-mon[30361]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:17:22.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:21 smithi204 ceph-mon[35013]: pgmap v357: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 14 KiB/s wr, 243 op/s 2024-09-12T03:17:22.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:21 smithi204 ceph-mon[35013]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T03:17:22.788 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-12T03:17:22.788 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-12T03:17:22.789 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-12T03:17:22.789 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=12 2024-09-12T03:17:22.789 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-12T03:17:22.789 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:17:22.789 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:17:22.789 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:17:22.789 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:17:23.173 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:23.836 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003220574, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000476633, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:23.837 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 250, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 238, 2024-09-12T03:17:23.838 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1726368, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 146624, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.059712444, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.019141495, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.058344793, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 237, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.031831018, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000134308 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.839 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1726368, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 146624 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 6, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 185, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 3392, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 190016, 2024-09-12T03:17:23.840 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.031534870, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.021123966, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.006175094, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 183, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.057753600, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000315593 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:23.841 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 190016 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:23.842 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:23.843 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:23.844 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 149745, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 31, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:23.845 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-09-12T03:17:23.846 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 252, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1691123, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 252, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 252, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1691123, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:23.847 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:23.848 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:23.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:23.848 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:23.848 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:17:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:23 smithi181 ceph-mon[35320]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 327 op/s 2024-09-12T03:17:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:23 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:23 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.349 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:23 smithi057 ceph-mon[30361]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 327 op/s 2024-09-12T03:17:24.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:23 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.350 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:23 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:23 smithi204 ceph-mon[35013]: pgmap v359: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 327 op/s 2024-09-12T03:17:24.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:23 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:24.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.394 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:23 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:17:24.752 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T03:17:24.752 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:17:24.753 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:17:25.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:24 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:25.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:24 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:25.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:24 smithi057 ceph-mon[30361]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:25.128 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:25.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:24 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:25.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:24 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:25.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:24 smithi204 ceph-mon[35013]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:25.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:24 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:25.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:24 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:25.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:24 smithi181 ceph-mon[35320]: from='mgr.15126 ' entity='mgr.x' 2024-09-12T03:17:25.730 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:17:25.730 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:25.730 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:25.730 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:25.730 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003844103, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000721310, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.731 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 249, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 235, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1728016, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 144352, 2024-09-12T03:17:25.732 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.092845139, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.037630068, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.069152638, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 235, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.077041408, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000327835 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:25.733 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1728016, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 144352 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 179, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 188864, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:25.734 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.025388048, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.018292747, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.002835407, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 176, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.036880414, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000209547 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.735 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 188864 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:25.736 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:25.737 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-09-12T03:17:25.738 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 148714, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 30, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:25.739 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:25.740 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 247, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1690409, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 247, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 247, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1690409, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:25.741 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:25.742 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:25.742 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:25.742 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:25.742 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:17:26.354 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:25 smithi181 ceph-mon[35320]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 250 op/s 2024-09-12T03:17:26.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:25 smithi057 ceph-mon[30361]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 250 op/s 2024-09-12T03:17:26.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:25 smithi204 ceph-mon[35013]: pgmap v360: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 250 op/s 2024-09-12T03:17:26.548 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:17:26.549 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T03:17:26.549 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:17:26.549 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:17:26.927 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout:{ 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:27.560 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.003120382, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.000535076, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:27.561 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 2, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 185, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 189920, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.026187308, 2024-09-12T03:17:27.562 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.019205201, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.003058935, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 182, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.043913716, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000241284 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:17:27.563 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 189920 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_messages": 256, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages": 242, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_bytes": 1729984, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_bytes": 148352, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_total_time": 0.067512772, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_send_time": 0.020578197, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_recv_time": 0.067224402, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:27.564 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 242, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.057043778, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000235718 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_recv_encrypted_bytes": 1729984, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "msgr_send_encrypted_bytes": 148352 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.565 INFO:teuthology.orchestra.run.smithi057.stdout: "cct": { 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "total_workers": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "unhealthy_workers": 0 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "mempool": { 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:27.566 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:27.567 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:27.568 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_bytes": 149745, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_anon_items": 31, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_items": 0, 2024-09-12T03:17:27.569 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "pgmap_items": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "mds_co_items": 2, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_1_items": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "unittest_2_items": 0 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: }, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "val": 0, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "max": 104857600, 2024-09-12T03:17:27.570 INFO:teuthology.orchestra.run.smithi057.stdout: "get_started": 0, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "get": 254, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "get_sum": 1691383, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "get_or_fail_success": 254, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "take": 0, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "take_sum": 0, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "put": 254, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "put_sum": 1691383, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "wait": { 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "avgcount": 0, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "sum": 0.000000000, 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: "avgtime": 0.000000000 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout: } 2024-09-12T03:17:27.571 INFO:teuthology.orchestra.run.smithi057.stdout:} 2024-09-12T03:17:28.217 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:27 smithi057 ceph-mon[30361]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 212 op/s 2024-09-12T03:17:28.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:27 smithi204 ceph-mon[35013]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 212 op/s 2024-09-12T03:17:28.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:27 smithi181 ceph-mon[35320]: pgmap v361: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 212 op/s 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:17:28.571 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T03:17:28.571 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:17:28.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:17:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:29.411 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-12T03:17:29.411 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:29.411 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2149, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1336, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 940637, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 2826294, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 11, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 4, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.217786712, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.057202049, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.100288724, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.059440238, 2024-09-12T03:17:29.412 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1338, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.353593774, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000264270 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2073, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000412377, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000198 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.413 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 409408, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 110656 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 929, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 879, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 2002521, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 2008326, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 113, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 6, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.137480683, 2024-09-12T03:17:29.414 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.033970503, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.092826983, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.016869525, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 877, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.382197098, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000435800 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 401, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000056781, 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000141 2024-09-12T03:17:29.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1914880, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 527856 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 1997, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1296, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 783895, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 911719, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 16, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 7, 2024-09-12T03:17:29.416 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.189445645, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.048819033, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.095987253, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.054393555, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1291, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.340778132, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000263964 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1518, 2024-09-12T03:17:29.417 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000374912, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000246 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 43632, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 151696 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-12T03:17:29.418 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1196, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.150071248, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000125477 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000013987, 2024-09-12T03:17:29.419 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000006993 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 226, 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 226, 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 226, 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.318210874, 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001408012 2024-09-12T03:17:29.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 187, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 35, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 7, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 17, 2024-09-12T03:17:29.421 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 11, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 25, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 24, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 3, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 3, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 3, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 288, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 273, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 1, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:29.422 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 1, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 20, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 1, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 4, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-09-12T03:17:29.423 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 2, 2024-09-12T03:17:29.424 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 16, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 103, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 107, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 85, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 2, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 24 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-12T03:17:29.425 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 7, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 3, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 2, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 2, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 16, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 2, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 2, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-12T03:17:29.426 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 1, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:29.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 331, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 321, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 321, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 10, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 10, 2024-09-12T03:17:29.428 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 52, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 51, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 51, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 1, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 1, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 1, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 6219548, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 6338029, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 0, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 271, 2024-09-12T03:17:29.429 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.584872221, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002158200 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 0 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 27, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 49, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 22, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 21, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 25, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 4, 2024-09-12T03:17:29.430 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 35, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 59, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 24, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 3, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 41, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 38, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 55636, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 231712 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 289, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 4, 2024-09-12T03:17:29.431 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 226, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 42, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 12, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 22, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.021207708, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000963986 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 134, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.026146457, 2024-09-12T03:17:29.432 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000195122 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.433 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 11, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.072079687, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006552698 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-09-12T03:17:29.434 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000349658, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000349658 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 12, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002005978, 2024-09-12T03:17:29.436 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000167164 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 10, 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.015798592, 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001579859 2024-09-12T03:17:29.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 12, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.074155299, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006179608 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.438 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000172289, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000172289 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.006093481, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.003046740 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-12T03:17:29.439 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002668634, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.001334317 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.035987718, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.017993859 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 6, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.037380406, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.006230067 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:29.441 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.001204911, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000602455 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.442 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.443 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 9, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.022960056, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.002551117 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.444 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.445 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 7, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 5, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 2, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 29, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 14, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 689, 2024-09-12T03:17:29.446 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 120, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 120, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:29.447 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:29.448 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:29.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 349561, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 132, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 352, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 4, 2024-09-12T03:17:29.450 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9784, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 197, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-12T03:17:29.451 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 149072, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 1077, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 1285, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 2749001, 2024-09-12T03:17:29.452 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 32, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 1285, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1285, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 28.136403821, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.021896034 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1285, 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1765 2024-09-12T03:17:29.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 1285, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 0, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 1285, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 165, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 66, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 0, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 271, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 166, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-12T03:17:29.454 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 506, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 110, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:29.455 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-12T03:17:29.456 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-12T03:17:29.457 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 110, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 25, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 12, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 12, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 467, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 0, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 14 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.458 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 1, 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-09-12T03:17:29.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:29.460 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 5071, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 3207178, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 5071, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 5071, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 3207178, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.461 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:29.462 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 1285, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2749001, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1285, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2749001, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:29.463 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 1285, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 1285, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 1285, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1285, 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:29.464 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:29.465 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x5556ec19c960": { 2024-09-12T03:17:29.466 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 331, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 2143725, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 331, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 271, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2143725, 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:29.467 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:29.468 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-12T03:17:30.340 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:29 smithi181 ceph-mon[35320]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 50 op/s 2024-09-12T03:17:30.368 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-09-12T03:17:30.368 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-12T03:17:30.368 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=7 2024-09-12T03:17:30.369 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3 2024-09-12T03:17:30.369 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-12T03:17:30.369 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=12 2024-09-12T03:17:30.369 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=6 2024-09-12T03:17:30.369 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:17:30.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:17:30.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:29 smithi057 ceph-mon[30361]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 50 op/s 2024-09-12T03:17:30.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:29 smithi204 ceph-mon[35013]: pgmap v362: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 50 op/s 2024-09-12T03:17:30.626 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 252, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 240, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1726912, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 147904, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:31.181 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.044959723, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.015972288, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.043561475, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 239, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.013403490, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000056081 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.182 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1726912, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 147904 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 6, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 185, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 3392, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 190016, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.013547465, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.007969643, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.003416125, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:31.183 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 183, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.015398046, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000084142 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 190016 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:31.184 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.000261585, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000184796, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:31.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-12T03:17:31.186 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:31.187 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:31.188 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 148754, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 31, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-12T03:17:31.189 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:31.190 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 254, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1691383, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 254, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 254, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1691383, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:31.191 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:31.192 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:31.192 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:31.192 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:31.192 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-12T03:17:32.250 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:31 smithi181 ceph-mon[35320]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 36 op/s 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-12T03:17:32.252 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-12T03:17:32.252 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:17:32.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:17:32.369 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:31 smithi057 ceph-mon[30361]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 36 op/s 2024-09-12T03:17:32.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:31 smithi204 ceph-mon[35013]: pgmap v363: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.2 KiB/s wr, 36 op/s 2024-09-12T03:17:32.505 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 3, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 5, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 795, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 960, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 6, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 2, 2024-09-12T03:17:33.125 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.001885351, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000059246, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000840556, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000069078, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 3, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.002149858, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000716619 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2045, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2026, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 4449435, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 712081, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.186696783, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.070089517, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.077599764, 2024-09-12T03:17:33.127 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.042050778, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2025, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.071239802, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000035180 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1784128, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 150432 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 825, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 1004, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 205523, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 732436, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 7, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 5, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.075581399, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.024869294, 2024-09-12T03:17:33.129 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.017895572, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.025281208, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 1002, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.046103876, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000046011 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.130 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 471984 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 1, 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-MDSRank": { 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-12T03:17:33.131 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 4333, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.104602575, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000024140 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "queue_len": 0, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "complete_latency": { 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.132 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "mds": { 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "request": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "reply": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "reply_latency": { 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_reply": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "forward": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:33.133 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_commit": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_split": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_merge": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes": 23, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_top": 5, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_bottom": 18, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pin_tail": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_pinned": 16, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_expired": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "inodes_with_caps": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "subtrees": 2, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse": 0, 2024-09-12T03:17:33.134 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_hit": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_forward": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_discover": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "traverse_lock": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "load_cent": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "q": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "exported": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "exported_inodes": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "imported": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "imported_inodes": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:33.135 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rfiles": 4, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rbytes": 234, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "root_rsnaps": 6, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:33.136 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_cap_release": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "process_request_cap_release": 0 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_cache": { 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_update_receipt": 0, 2024-09-12T03:17:33.137 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_try_discover": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_send_discover": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "dir_handle_discover": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_created": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_enqueued": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "strays_migrated": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:33.138 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_started": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "recovery_completed": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_exportdir": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_flush": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "ireq_inodestats": 0 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_log": { 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "evlrg": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "evadd": 0, 2024-09-12T03:17:33.139 INFO:teuthology.orchestra.run.smithi181.stdout: "evex": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "evtrm": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "ev": 331, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "evexg": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "evexd": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segadd": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segex": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segtrm": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "seg": 2, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segmjr": 52, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segexg": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "segexd": 0, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "expos": 6219548, 2024-09-12T03:17:33.140 INFO:teuthology.orchestra.run.smithi181.stdout: "wrpos": 6338029, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "rdpos": 6337968, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "jlat": { 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "replayed": 331 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_mem": { 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "ino": 17, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "ino+": 37, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "ino-": 20, 2024-09-12T03:17:33.141 INFO:teuthology.orchestra.run.smithi181.stdout: "dir": 11, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dir+": 13, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dir-": 2, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dn": 23, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dn+": 43, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dn-": 20, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "cap": 0, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "cap+": 0, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "cap-": 0, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "rss": 48636, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "heap": 215328 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_server": { 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_client_request": 0, 2024-09-12T03:17:33.142 INFO:teuthology.orchestra.run.smithi181.stdout: "dispatch_server_request": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_request": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_client_session": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "handle_peer_request": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "req_create_latency": { 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getattr_latency": { 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.143 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "req_link_latency": { 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookup_latency": { 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.144 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupino_latency": { 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupname_latency": { 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.145 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "req_lssnap_latency": { 2024-09-12T03:17:33.146 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mkdir_latency": { 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mknod_latency": { 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.147 INFO:teuthology.orchestra.run.smithi181.stdout: "req_mksnap_latency": { 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "req_open_latency": { 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "req_readdir_latency": { 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.148 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rename_latency": { 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:33.149 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmdir_latency": { 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setattr_latency": { 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.151 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setlayout_latency": { 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "req_setxattr_latency": { 2024-09-12T03:17:33.152 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "req_symlink_latency": { 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "req_unlink_latency": { 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_sessions": { 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "session_count": 2, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "session_add": 7, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "session_remove": 5, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_open": 5, 2024-09-12T03:17:33.154 INFO:teuthology.orchestra.run.smithi181.stdout: "sessions_stale": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "total_load": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "average_load": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "avg_session_uptime": 4310, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:33.155 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:33.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:33.157 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 198740, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 70, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 9784, 2024-09-12T03:17:33.158 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 197, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 103001, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 723, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "objecter": { 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "op_active": 0, 2024-09-12T03:17:33.159 INFO:teuthology.orchestra.run.smithi181.stdout: "op_laggy": 0, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send": 2588, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_send_bytes": 26, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_resend": 4, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_reply": 2588, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_latency": { 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2588, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 1.023978347, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000395663 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "op_inflight": 0, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "oplen_avg": { 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 2588, 2024-09-12T03:17:33.160 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 2589 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op": 2588, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_r": 2588, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_w": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_rmw": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_pg": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_stat": 1632, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_create": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_read": 953, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_write": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writefull": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_writesame": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_append": 0, 2024-09-12T03:17:33.161 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_zero": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_truncate": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_delete": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_mapext": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_getxattr": 2, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_setxattr": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_call": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_watch": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_notify": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:33.162 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "osdop_other": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_active": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_send": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_resend": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "linger_ping": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_active": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_send": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolop_resend": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_active": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_send": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "poolstat_resend": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_active": 0, 2024-09-12T03:17:33.163 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_send": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "statfs_resend": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "command_active": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "command_send": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "command_resend": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "map_epoch": 110, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "map_full": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "map_inc": 16, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_sessions": 6, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_open": 6, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_session_close": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_laggy": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_wr": 0, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_rd": 2, 2024-09-12T03:17:33.164 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_del": 0 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "oft": { 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_objs": 0, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_updates": 0, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "omap_total_removes": 0 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "osd-slow-ops": { 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "slow_ops_count": 0 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "purge_queue": { 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:33.165 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executing_high_water": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed_ops": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_executed": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 2869, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 4380230, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 2869, 2024-09-12T03:17:33.166 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2869, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 4380230, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:33.167 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2588, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2143735, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2588, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2143735, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.168 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 1024, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 2588, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 2588, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 2588, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 2588, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.169 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:33.170 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-write_buf_throttle-0x55b42cca6b40": { 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 3758096384, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:33.171 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:33.172 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-12T03:17:33.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:32 smithi057 ceph-mon[30361]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-12T03:17:33.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:32 smithi204 ceph-mon[35013]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-12T03:17:33.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:32 smithi181 ceph-mon[35320]: pgmap v364: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.0 KiB/s wr, 17 op/s 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:17:33.989 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:17:33.989 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:17:33.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:17:34.236 INFO:teuthology.orchestra.run.smithi181.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:17:34.833 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.001032652, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:34.834 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.000225544, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:34.835 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 2, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 186, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 190096, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.011612249, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.007891885, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.001269052, 2024-09-12T03:17:34.836 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 183, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.020318034, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000111027 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 190096 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.837 INFO:teuthology.orchestra.run.smithi181.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_messages": 260, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages": 246, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_bytes": 1731696, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_bytes": 150608, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_total_time": 0.047461830, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_send_time": 0.016573255, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_recv_time": 0.045667772, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 246, 2024-09-12T03:17:34.838 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.017327383, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000070436 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_recv_encrypted_bytes": 1731696, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "msgr_send_encrypted_bytes": 150608 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "cct": { 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "total_workers": 0, 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: "unhealthy_workers": 0 2024-09-12T03:17:34.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "mempool": { 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:34.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:34.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_bytes": 148709, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_anon_items": 31, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:34.842 INFO:teuthology.orchestra.run.smithi181.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "pgmap_items": 0, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:34.843 INFO:teuthology.orchestra.run.smithi181.stdout: "mds_co_items": 2, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_1_items": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "unittest_2_items": 0 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "max": 104857600, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "get_started": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "get": 258, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "get_sum": 1692540, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "get_or_fail_success": 258, 2024-09-12T03:17:34.844 INFO:teuthology.orchestra.run.smithi181.stdout: "take": 0, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "take_sum": 0, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "put": 258, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "put_sum": 1692540, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "wait": { 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "avgcount": 0, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "sum": 0.000000000, 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: "avgtime": 0.000000000 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-12T03:17:34.845 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-12T03:17:35.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:35 smithi181 ceph-mon[35320]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:35.600 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T03:17:35.600 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T03:17:35.600 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-12T03:17:35.601 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-12T03:17:35.601 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T03:17:35.601 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:17:35.601 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T03:17:35.601 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:17:35.601 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:17:35.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:35 smithi057 ceph-mon[30361]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:35.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:35 smithi204 ceph-mon[35013]: pgmap v365: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:36.402 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-12T03:17:36.402 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:36.402 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1098, 2024-09-12T03:17:36.402 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1079, 2024-09-12T03:17:36.402 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2059424, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 411354, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.135825937, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.043888073, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.070662598, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.028660787, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1078, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.056319495, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000052244 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:36.403 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1784240, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 150912 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 8, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 187, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3948, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 471066, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-12T03:17:36.404 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.012943442, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.007170263, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.003760262, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000057358, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 185, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.023545911, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000127275 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 470368 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1545, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1547, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 396179, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 491317, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.101505852, 2024-09-12T03:17:36.406 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.032398891, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.025673519, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.036404941, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1545, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.061697267, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000039933 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:36.407 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:36.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3996, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.102150445, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000025563 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.409 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-12T03:17:36.410 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-12T03:17:36.411 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:36.412 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:17:36.413 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-12T03:17:36.414 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-12T03:17:36.415 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-12T03:17:36.416 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 8, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196664, 2024-09-12T03:17:36.417 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4196523, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 8 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-09-12T03:17:36.418 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 49004, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-12T03:17:36.419 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.420 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.421 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.422 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.423 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-12T03:17:36.424 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.425 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.426 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:36.427 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.428 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.429 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:36.430 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 4, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-12T03:17:36.431 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 2830, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:36.432 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:36.433 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 182098, 2024-09-12T03:17:36.434 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 49, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9784, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 197, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:36.435 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8512, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2366, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-12T03:17:36.436 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2366, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2366, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.209982973, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000511404 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2366, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2367 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2366, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2366, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-12T03:17:36.437 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1535, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 829, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:36.438 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-12T03:17:36.439 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-12T03:17:36.440 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 110, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 4, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 4, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-12T03:17:36.441 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-12T03:17:36.442 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2647, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2204002, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2647, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:36.443 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2647, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2204002, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:36.444 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2366, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2365, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2366, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2365, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:36.445 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2366, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2366, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2366, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2366, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.446 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-12T03:17:36.447 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x5581e76e6b40": { 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:36.448 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:36.449 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:17:37.244 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:17:37.244 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:17:37.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:17:37.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:37 smithi204 ceph-mon[35013]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:37.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:37 smithi181 ceph-mon[35320]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:37.541 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:37.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:37 smithi057 ceph-mon[30361]: pgmap v366: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 283, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 264, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1786514, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 152852, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.047162791, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.016793609, 2024-09-12T03:17:38.063 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.045721067, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000016088, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 263, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.013541989, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000051490 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1785680, 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 152528 2024-09-12T03:17:38.064 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 701, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 884, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 172296, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 695791, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.078137380, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.024178622, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.018221791, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.027178262, 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:38.065 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 881, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.043449767, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000049318 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 473504 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1738, 2024-09-12T03:17:38.066 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1738, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3738691, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 551918, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.168023261, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.049722453, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.049985928, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.054081031, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1737, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.070642034, 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000040668 2024-09-12T03:17:38.067 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-12T03:17:38.068 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4070, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.116099082, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000028525 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.069 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:38.070 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 104, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 104, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 104, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-12T03:17:38.071 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:38.072 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:17:38.073 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-12T03:17:38.074 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:38.075 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-12T03:17:38.076 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 926, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-12T03:17:38.077 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 7416658, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 7413771, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 926 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 70, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 97, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 27, 2024-09-12T03:17:38.078 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 4, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 13, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 9, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 104, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 116, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 12, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 50036, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-12T03:17:38.079 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:38.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.081 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.082 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.083 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-12T03:17:38.083 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.083 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.083 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.084 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.085 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-12T03:17:38.086 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.087 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:38.088 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-12T03:17:38.089 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.090 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.091 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.092 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:38.093 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 4, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-12T03:17:38.094 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 6385, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:38.095 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:38.096 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 192760, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 124, 2024-09-12T03:17:38.097 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9784, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 197, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-12T03:17:38.098 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 247441, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2247, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2434, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2434, 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-12T03:17:38.099 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2434, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 5.097927783, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002094464 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2434, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2435 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2434, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2434, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-12T03:17:38.100 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1514, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 918, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:38.101 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-12T03:17:38.102 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-12T03:17:38.103 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 110, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-12T03:17:38.104 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:38.105 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2718, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 5435419, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:38.106 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2718, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2718, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 5435419, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:38.107 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2434, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3222359, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2434, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3222359, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.108 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2434, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2434, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2434, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2434, 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:38.109 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:38.110 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.111 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x558f7a042b40": { 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:38.112 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:38.113 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-12T03:17:38.921 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:17:38.921 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:17:38.921 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T03:17:38.921 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T03:17:38.921 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-12T03:17:38.922 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:17:38.922 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:17:38.922 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:17:38.922 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:17:39.206 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:39.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:39 smithi181 ceph-mon[35320]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:17:39.460 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:39 smithi204 ceph-mon[35013]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:17:39.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:39 smithi057 ceph-mon[30361]: pgmap v367: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2774, 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 3082, 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 837036, 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 3659476, 2024-09-12T03:17:39.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.220188108, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.075531307, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.110896416, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.042633611, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3082, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.137391686, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000044578 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1812, 2024-09-12T03:17:39.677 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000555258, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000306 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 978, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1161, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 251256, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 848419, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 11, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 7, 2024-09-12T03:17:39.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.086753643, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.029151043, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.025985495, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.024819918, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1159, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.072307496, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000062387 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 16, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000002419, 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000151 2024-09-12T03:17:39.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 5248, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 476944 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 848, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 857, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1940337, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 433147, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 14, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 7, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.093775942, 2024-09-12T03:17:39.680 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.029922911, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.062447762, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.013547236, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 854, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.054578380, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000063909 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 82, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000012833, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000156 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.681 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1796224, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 159648 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 382, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.192675427, 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000504385 2024-09-12T03:17:39.682 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2511, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.089358439, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000035586 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 49, 2024-09-12T03:17:39.683 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 47, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 47, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.930376092, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.083625023 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 2, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-12T03:17:39.684 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 99, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 85, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 14, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 98, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 8, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 88, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 76, 2024-09-12T03:17:39.685 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 2, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 3, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 1, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 4, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 1, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 20, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:17:39.686 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 2, 2024-09-12T03:17:39.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 48, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 116, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 839, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 778, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 4, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 39 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 2, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 11, 2024-09-12T03:17:39.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 7, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 8, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 4, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 2, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 20, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 18, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:17:39.689 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 1, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-12T03:17:39.690 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 926, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 926, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-12T03:17:39.691 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 7416658, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 151, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.351349894, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002326820 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 85, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 107, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 22, 2024-09-12T03:17:39.692 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 20, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 25, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 5, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 101, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 127, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 26, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 26, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 26, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 57572, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223520 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-12T03:17:39.693 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 67, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 1, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 49, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 22, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 6, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.003369707, 2024-09-12T03:17:39.694 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001684853 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 22, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.009546185, 2024-09-12T03:17:39.695 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000433917 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-12T03:17:39.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.697 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-12T03:17:39.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001962911, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000654303 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.700 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.701 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.890845442, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 1.945422721 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.703 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 18, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.024651847, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001369547 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 4, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 36, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 36, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 699, 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-12T03:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:39.707 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:39.708 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 309445, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 139, 2024-09-12T03:17:39.709 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9784, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 197, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-12T03:17:39.710 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 296066, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2095, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2832, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 3241527, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 28, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2832, 2024-09-12T03:17:39.711 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2832, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 60.921160900, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.021511709 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2832, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2872 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2832, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 1, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 2831, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-12T03:17:39.712 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 152, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 31, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 2608, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-12T03:17:39.713 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-12T03:17:39.714 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-12T03:17:39.715 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 110, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 39, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 11 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-12T03:17:39.716 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-12T03:17:39.717 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 524, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 6, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 2596, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 18, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 4596, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2609987, 2024-09-12T03:17:39.718 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 4596, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 4596, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2609987, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:39.719 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2832, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 3244353, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2832, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3244353, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.720 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2832, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2832, 2024-09-12T03:17:39.721 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2832, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2832, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 18, 2024-09-12T03:17:39.722 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2826, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 18, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2826, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x556496360960": { 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:39.723 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 926, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 3222354, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 926, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 151, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 3222354, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:39.724 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.725 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:39.725 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-09-12T03:17:40.474 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-12T03:17:40.475 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T03:17:40.475 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:17:40.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:17:40.720 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:17:41.327 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 262, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 248, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1732240, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 151888, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.046966982, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.015696571, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.044607315, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:41.328 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 248, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.016485996, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000066475 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1732240, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 151888 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.329 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 188, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 190448, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.011976411, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008234284, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001361445, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 185, 2024-09-12T03:17:41.330 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012645736, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000068355 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 190448 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 0, 2024-09-12T03:17:41.331 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 2, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 0, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.000269443, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000193922, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:41.332 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.333 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:17:41.334 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:17:41.335 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 148754, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 31, 2024-09-12T03:17:41.336 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-12T03:17:41.337 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 260, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1692800, 2024-09-12T03:17:41.338 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 260, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 260, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1692800, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-12T03:17:41.339 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-12T03:17:41.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:41 smithi204 ceph-mon[35013]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:41.453 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:41 smithi181 ceph-mon[35320]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:41.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:41 smithi057 ceph-mon[30361]: pgmap v368: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:17:42.145 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-12T03:17:42.145 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:17:42.156 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:17:42.156 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:17:42.157 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:42.212 DEBUG:teuthology.orchestra.run.smithi057:> set -ex 2024-09-12T03:17:42.213 DEBUG:teuthology.orchestra.run.smithi057:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:17:42.292 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:17:42.718 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:43.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 -- 172.21.15.57:0/1948205451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b010bfc0 msgr2=0x7ff4b010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.331 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 --2- 172.21.15.57:0/1948205451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b010bfc0 0x7ff4b010e460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009ca0 tx=0x7ff4a802a950 comp rx=0 tx=0).stop 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 -- 172.21.15.57:0/1948205451 shutdown_connections 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 --2- 172.21.15.57:0/1948205451 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b010bfc0 0x7ff4b010e460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 --2- 172.21.15.57:0/1948205451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b01034b0 0x7ff4b010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 --2- 172.21.15.57:0/1948205451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0102b90 0x7ff4b0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.330+0000 7ff4afe00640 1 -- 172.21.15.57:0/1948205451 >> 172.21.15.57:0/1948205451 conn(0x7ff4b00fc610 msgr2=0x7ff4b00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.331+0000 7ff4afe00640 1 -- 172.21.15.57:0/1948205451 shutdown_connections 2024-09-12T03:17:43.332 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.331+0000 7ff4afe00640 1 -- 172.21.15.57:0/1948205451 wait complete. 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.331+0000 7ff4afe00640 1 Processor -- start 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 -- start start 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 0x7ff4b019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b010bfc0 0x7ff4b019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0111df0 con 0x7ff4b01034b0 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0111c70 con 0x7ff4b010bfc0 2024-09-12T03:17:43.333 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4afe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0111f70 con 0x7ff4b0102b90 2024-09-12T03:17:43.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 0x7ff4b019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:57584/0 (socket says 172.21.15.57:57584) 2024-09-12T03:17:43.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.332+0000 7ff4aea00640 1 -- 172.21.15.57:0/366994629 learned_addr learned my addr 172.21.15.57:0/366994629 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:43.334 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4af400640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b010bfc0 0x7ff4b019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 -- 172.21.15.57:0/366994629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b010bfc0 msgr2=0x7ff4b019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b010bfc0 0x7ff4b019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 -- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 msgr2=0x7ff4b019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 0x7ff4b019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 -- 172.21.15.57:0/366994629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b019f430 con 0x7ff4b0102b90 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4a7e00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 0x7ff4b019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.333+0000 7ff4aea00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000e9f0 tx=0x7ff4a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.334+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000cd80 con 0x7ff4b0102b90 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.334+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b019f720 con 0x7ff4b0102b90 2024-09-12T03:17:43.335 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.334+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0010910 con 0x7ff4b0102b90 2024-09-12T03:17:43.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.334+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00188f0 con 0x7ff4b0102b90 2024-09-12T03:17:43.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.334+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b01a7000 con 0x7ff4b0102b90 2024-09-12T03:17:43.336 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.335+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b0102f70 con 0x7ff4b0102b90 2024-09-12T03:17:43.337 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.337+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7ff4a0010ac0 con 0x7ff4b0102b90 2024-09-12T03:17:43.338 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.338+0000 7ff4a7400640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:43.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.338+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9538+0+0 (secure 0 0 0) 0x7ff4a00d0120 con 0x7ff4b0102b90 2024-09-12T03:17:43.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.338+0000 7ff4a7e00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:43.339 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.339+0000 7ff4a7e00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 0x7ff4940a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4b010e920 tx=0x7ff498005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:43.340 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.340+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4a009c290 con 0x7ff4b0102b90 2024-09-12T03:17:43.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.538+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4b01a72a0 con 0x7ff4b0102b90 2024-09-12T03:17:43.539 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:43 smithi057 ceph-mon[30361]: pgmap v369: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:17:43.539 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.539+0000 7ff4a7400640 1 -- 172.21.15.57:0/366994629 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v110) ==== 81+0+941 (secure 0 0 0) 0x7ff4a00bae20 con 0x7ff4b0102b90 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2696288462 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/752811668 2024-09-13T02:55:39.619488+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1354301616 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1176788560 2024-09-13T02:55:39.619488+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1345986708 2024-09-13T02:55:10.466284+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1679473944 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/522425633 2024-09-13T02:55:39.619488+0000 2024-09-12T03:17:43.540 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/2870728447 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/1326814975 2024-09-13T02:55:10.466284+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6801/480086157 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1256058423 2024-09-13T02:55:10.466284+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/1578458239 2024-09-13T02:55:39.619488+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:0/4021182045 2024-09-13T02:55:10.466284+0000 2024-09-12T03:17:43.541 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57:6800/480086157 2024-09-13T03:05:55.702194+0000 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:listed 16 entries 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 msgr2=0x7ff4940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 0x7ff4940a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4b010e920 tx=0x7ff498005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 msgr2=0x7ff4b019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000e9f0 tx=0x7ff4a000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 shutdown_connections 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b010bfc0 0x7ff4b019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7ff4b01034b0 0x7ff4b019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 --2- 172.21.15.57:0/366994629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4b0102b90 0x7ff4b019a480 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:43.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 >> 172.21.15.57:0/366994629 conn(0x7ff4b00fc610 msgr2=0x7ff4b010d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:43.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 shutdown_connections 2024-09-12T03:17:43.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:43.541+0000 7ff4afe00640 1 -- 172.21.15.57:0/366994629 wait complete. 2024-09-12T03:17:43.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:43 smithi204 ceph-mon[35013]: pgmap v369: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:17:43.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:43 smithi181 ceph-mon[35320]: pgmap v369: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-12T03:17:44.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:44 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/366994629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:17:44.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:44 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/366994629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:17:44.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:44 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/366994629' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:17:44.783 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:17:44.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.784 DEBUG:teuthology.orchestra.run.smithi057:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:17:44.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:17:44.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.881 DEBUG:teuthology.orchestra.run.smithi057:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:17:44.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:44.930 DEBUG:teuthology.orchestra.run.smithi057:> ip netns list 2024-09-12T03:17:45.011 INFO:teuthology.orchestra.run.smithi057.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:17:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.012 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:17:45.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:17:45.072 DEBUG:teuthology.orchestra.run.smithi057:> sudo ip link delete ceph-brx 2024-09-12T03:17:45.428 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:17:45.439 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:17:45.439 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:17:45.439 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:45.487 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:45 smithi057 ceph-mon[30361]: pgmap v370: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:45.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:45 smithi204 ceph-mon[35013]: pgmap v370: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:45.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:45 smithi181 ceph-mon[35320]: pgmap v370: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T03:17:45.862 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 -- 172.21.15.57:0/1293606955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90112a50 msgr2=0x7f7b90114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 --2- 172.21.15.57:0/1293606955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90112a50 0x7f7b90114e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009ca0 tx=0x7f7b8802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.441 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 -- 172.21.15.57:0/1293606955 shutdown_connections 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 --2- 172.21.15.57:0/1293606955 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90112a50 0x7f7b90114e40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 --2- 172.21.15.57:0/1293606955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90077040 0x7f7b90112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 --2- 172.21.15.57:0/1293606955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90076720 0x7f7b90076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 -- 172.21.15.57:0/1293606955 >> 172.21.15.57:0/1293606955 conn(0x7f7b900ffa50 msgr2=0x7f7b90101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 -- 172.21.15.57:0/1293606955 shutdown_connections 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.424+0000 7f7b96c00640 1 -- 172.21.15.57:0/1293606955 wait complete. 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.425+0000 7f7b96c00640 1 Processor -- start 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.425+0000 7f7b96c00640 1 -- start start 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 0x7f7b901a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90112a50 0x7f7b9019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90118590 con 0x7f7b90077040 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90118410 con 0x7f7b90076720 2024-09-12T03:17:46.442 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b96c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90118710 con 0x7f7b90112a50 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b8f400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52680/0 (socket says 172.21.15.57:52680) 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b8f400640 1 -- 172.21.15.57:0/437317149 learned_addr learned my addr 172.21.15.57:0/437317149 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b8fe00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 0x7f7b901a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.426+0000 7f7b94c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90112a50 0x7f7b9019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 -- 172.21.15.57:0/437317149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90112a50 msgr2=0x7f7b9019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90112a50 0x7f7b9019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 -- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 msgr2=0x7f7b901a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 -- 172.21.15.57:0/437317149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9019efb0 con 0x7f7b90076720 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8f400640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8fe00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 0x7f7b901a46c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9f0 tx=0x7f7b8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000ccc0 con 0x7f7b90076720 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80010910 con 0x7f7b90076720 2024-09-12T03:17:46.443 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80018960 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9019f2a0 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.427+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901ab590 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.428+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b90076b00 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.430+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f7b80010ac0 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.432+0000 7f7b8cc00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.432+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9538+0+0 (secure 0 0 0) 0x7f7b800cff40 con 0x7f7b90076720 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.432+0000 7f7b8f400640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.432+0000 7f7b8f400640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 0x7f7b740a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b90079290 tx=0x7f7b7c005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:46.444 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.434+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b8009c0b0 con 0x7f7b90076720 2024-09-12T03:17:46.660 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.659+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b901ab880 con 0x7f7b90076720 2024-09-12T03:17:46.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.660+0000 7f7b8cc00640 1 -- 172.21.15.57:0/437317149 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11924 (secure 0 0 0) 0x7f7b800a03b0 con 0x7f7b90076720 2024-09-12T03:17:46.661 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:17:46.661 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"btime":"2024-09-12T03:17:16:913432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:17:16.096469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:17:46.662 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-09-12T03:17:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 msgr2=0x7f7b740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.663 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 0x7f7b740a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b90079290 tx=0x7f7b7c005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 msgr2=0x7f7b901a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 0x7f7b901a46c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9f0 tx=0x7f7b8000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 shutdown_connections 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b90112a50 0x7f7b9019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f7b740a2cf0 0x7f7b740a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f7b90077040 0x7f7b901a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 --2- 172.21.15.57:0/437317149 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b90076720 0x7f7b901a46c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 >> 172.21.15.57:0/437317149 conn(0x7f7b900ffa50 msgr2=0x7f7b90101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 shutdown_connections 2024-09-12T03:17:46.664 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:46.663+0000 7f7b96c00640 1 -- 172.21.15.57:0/437317149 wait complete. 2024-09-12T03:17:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:47 smithi204 ceph-mon[35013]: pgmap v371: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:47.393 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:47 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/437317149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:47.457 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:47 smithi057 ceph-mon[30361]: pgmap v371: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:47.457 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:47 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/437317149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:47.497 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:47 smithi181 ceph-mon[35320]: pgmap v371: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-12T03:17:47.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:47 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/437317149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:47.899 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ecbe00640 1 -- 172.21.15.57:0/4243352833 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec402c040 con 0x7f5ecc10bfc0 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 -- 172.21.15.57:0/4243352833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc10bfc0 msgr2=0x7f5ecc10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/4243352833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc10e460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009ca0 tx=0x7f5ec402a950 comp rx=0 tx=0).stop 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 -- 172.21.15.57:0/4243352833 shutdown_connections 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/4243352833 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc10e460 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/4243352833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/4243352833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 0x7f5ecc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 -- 172.21.15.57:0/4243352833 >> 172.21.15.57:0/4243352833 conn(0x7f5ecc0fc610 msgr2=0x7f5ecc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 -- 172.21.15.57:0/4243352833 shutdown_connections 2024-09-12T03:17:48.500 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.499+0000 7f5ed2a00640 1 -- 172.21.15.57:0/4243352833 wait complete. 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.500+0000 7f5ed2a00640 1 Processor -- start 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.500+0000 7f5ed2a00640 1 -- start start 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 0x7f5ecc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111e10 con 0x7f5ecc1034b0 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111c90 con 0x7f5ecc10bfc0 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111f90 con 0x7f5ecc102b90 2024-09-12T03:17:48.502 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed0c00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58000/0 (socket says 172.21.15.57:58000) 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed0c00640 1 -- 172.21.15.57:0/498414192 learned_addr learned my addr 172.21.15.57:0/498414192 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed1600640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 0x7f5ecc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed2000640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed0c00640 1 -- 172.21.15.57:0/498414192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 msgr2=0x7f5ecc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.501+0000 7f5ed0c00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 0x7f5ecc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed0c00640 1 -- 172.21.15.57:0/498414192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc10bfc0 msgr2=0x7f5ecc19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed0c00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.503 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed0c00640 1 -- 172.21.15.57:0/498414192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19f3e0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed0c00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800e9f0 tx=0x7f5eb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800ccc0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb8010910 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8018920 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19f6d0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.502+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a6fb0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.504 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.503+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc102f70 con 0x7f5ecc1034b0 2024-09-12T03:17:48.505 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.505+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f5eb8010430 con 0x7f5ecc1034b0 2024-09-12T03:17:48.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.506+0000 7f5eca000640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 0x7f5eb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.507+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9538+0+0 (secure 0 0 0) 0x7f5eb80cfa80 con 0x7f5ecc1034b0 2024-09-12T03:17:48.507 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.507+0000 7f5ed1600640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 0x7f5eb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.508 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.507+0000 7f5ed1600640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 0x7f5eb40a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009bd0 tx=0x7f5ec0005f90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.511 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.511+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eb80d1e20 con 0x7f5ecc1034b0 2024-09-12T03:17:48.738 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.738+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ecc1a71e0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.739 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.739+0000 7f5eca000640 1 -- 172.21.15.57:0/498414192 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11924 (secure 0 0 0) 0x7f5eb809bbf0 con 0x7f5ecc1034b0 2024-09-12T03:17:48.740 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:17:48.740 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":31,"btime":"2024-09-12T03:17:16:913432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6839/1375693587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6838","nonce":1375693587},{"type":"v1","addr":"172.21.15.57:6839","nonce":1375693587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6833/1889756926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":1889756926},{"type":"v1","addr":"172.21.15.181:6833","nonce":1889756926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3741354654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3741354654},{"type":"v1","addr":"172.21.15.204:6833","nonce":3741354654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6835/4044082469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6834","nonce":4044082469},{"type":"v1","addr":"172.21.15.57:6835","nonce":4044082469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24479,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.57:6837/3595237430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6836","nonce":3595237430},{"type":"v1","addr":"172.21.15.57:6837","nonce":3595237430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1691601828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1691601828},{"type":"v1","addr":"172.21.15.181:6835","nonce":1691601828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:03:08.679612+0000","modified":"2024-09-12T03:17:16.096469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24554,"mds_1":24548,"mds_2":24529},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/1804109241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":1804109241},{"type":"v1","addr":"172.21.15.204:6835","nonce":1804109241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/368351857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":368351857},{"type":"v1","addr":"172.21.15.204:6837","nonce":368351857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.181:6837/2458916124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2458916124},{"type":"v1","addr":"172.21.15.181:6837","nonce":2458916124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24529":{"gid":24529,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.204:6839/3278241191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3278241191},{"type":"v1","addr":"172.21.15.204:6839","nonce":3278241191}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.57:6841/1479422251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6840","nonce":1479422251},{"type":"v1","addr":"172.21.15.57:6841","nonce":1479422251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24554":{"gid":24554,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.181:6839/399579254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6838","nonce":399579254},{"type":"v1","addr":"172.21.15.181:6839","nonce":399579254}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24554,"qdb_cluster":[24529,24548,24554]},"id":1}]} 2024-09-12T03:17:48.740 INFO:teuthology.orchestra.run.smithi057.stderr:dumped fsmap epoch 31 2024-09-12T03:17:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 msgr2=0x7f5eb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 0x7f5eb40a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009bd0 tx=0x7f5ec0005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 msgr2=0x7f5ecc19a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.742 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800e9f0 tx=0x7f5eb800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 shutdown_connections 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ecc10bfc0 0x7f5ecc19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f5eb40a2cf0 0x7f5eb40a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f5ecc1034b0 0x7f5ecc19a970 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 --2- 172.21.15.57:0/498414192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ecc102b90 0x7f5ecc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 >> 172.21.15.57:0/498414192 conn(0x7f5ecc0fc610 msgr2=0x7f5ecc0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 shutdown_connections 2024-09-12T03:17:48.743 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:48.742+0000 7f5ed2a00640 1 -- 172.21.15.57:0/498414192 wait complete. 2024-09-12T03:17:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:49 smithi057 ceph-mon[30361]: pgmap v372: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:17:49.534 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:49 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/498414192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:49.575 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:49.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:49 smithi204 ceph-mon[35013]: pgmap v372: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:17:49.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:49 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/498414192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:49 smithi181 ceph-mon[35320]: pgmap v372: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T03:17:49.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:49 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/498414192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:49.953 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:50.518 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.517+0000 7f62f6a00640 1 -- 172.21.15.57:0/2114895473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 msgr2=0x7f62f0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.517+0000 7f62f6a00640 1 --2- 172.21.15.57:0/2114895473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f0108a10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009900 tx=0x7f62dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 -- 172.21.15.57:0/2114895473 shutdown_connections 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 --2- 172.21.15.57:0/2114895473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0109140 0x7f62f0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 --2- 172.21.15.57:0/2114895473 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f0108a10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 --2- 172.21.15.57:0/2114895473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0104010 0x7f62f01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 -- 172.21.15.57:0/2114895473 >> 172.21.15.57:0/2114895473 conn(0x7f62f00ffa50 msgr2=0x7f62f0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 -- 172.21.15.57:0/2114895473 shutdown_connections 2024-09-12T03:17:50.519 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.518+0000 7f62f6a00640 1 -- 172.21.15.57:0/2114895473 wait complete. 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 Processor -- start 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 -- start start 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0109140 0x7f62f019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114140 con 0x7f62f01049c0 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0113fc0 con 0x7f62f0104010 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.519+0000 7f62f6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01142c0 con 0x7f62f0109140 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.521 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38108/0 (socket says 172.21.15.57:38108) 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 -- 172.21.15.57:0/4285829638 learned_addr learned my addr 172.21.15.57:0/4285829638 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62ef400640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62f4a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0109140 0x7f62f019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 -- 172.21.15.57:0/4285829638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0109140 msgr2=0x7f62f019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0109140 0x7f62f019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 -- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 msgr2=0x7f62f019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 -- 172.21.15.57:0/4285829638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f019f660 con 0x7f62f0104010 2024-09-12T03:17:50.522 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.520+0000 7f62efe00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e9f0 tx=0x7f62e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.521+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e000cd80 con 0x7f62f0104010 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.521+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f019f950 con 0x7f62f0104010 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.521+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e0010910 con 0x7f62f0104010 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.521+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e00188f0 con 0x7f62f0104010 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.521+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a7230 con 0x7f62f0104010 2024-09-12T03:17:50.523 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.522+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f01043f0 con 0x7f62f0104010 2024-09-12T03:17:50.524 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.524+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f62e0010ac0 con 0x7f62f0104010 2024-09-12T03:17:50.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.525+0000 7f62ecc00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 0x7f62d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.526+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9538+0+0 (secure 0 0 0) 0x7f62e00cffe0 con 0x7f62f0104010 2024-09-12T03:17:50.526 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.526+0000 7f62ef400640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 0x7f62d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.526+0000 7f62ef400640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 0x7f62d40a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009230 tx=0x7f62dc0058d0 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.527 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.527+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62e009c1b0 con 0x7f62f0104010 2024-09-12T03:17:50.713 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.712+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62f010c380 con 0x7f62f0104010 2024-09-12T03:17:50.715 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.714+0000 7f62ecc00640 1 -- 172.21.15.57:0/4285829638 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+25450 (secure 0 0 0) 0x7f62e00a04b0 con 0x7f62f0104010 2024-09-12T03:17:50.715 INFO:teuthology.orchestra.run.smithi057.stdout: 2024-09-12T03:17:50.716 INFO:teuthology.orchestra.run.smithi057.stdout:{"epoch":110,"fsid":"3c25d9c4-70b2-11ef-bcea-c7b262605968","created":"2024-09-12T02:54:45.449832+0000","modified":"2024-09-12T03:17:21.659844+0000","last_up_change":"2024-09-12T03:02:11.632076+0000","last_in_change":"2024-09-12T03:01:47.353406+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:58:06.250728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:03:03.359994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:03:06.244623+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-12T03:03:16.018628+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32912ece-df23-430a-95fe-259e65d98666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6802","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6803","nonce":3978791581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6804","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6805","nonce":3978791581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6808","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6809","nonce":3978791581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6806","nonce":3978791581},{"type":"v1","addr":"172.21.15.57:6807","nonce":3978791581}]},"public_addr":"172.21.15.57:6803/3978791581","cluster_addr":"172.21.15.57:6805/3978791581","heartbeat_back_addr":"172.21.15.57:6809/3978791581","heartbeat_front_addr":"172.21.15.57:6807/3978791581","state":["exists","up"]},{"osd":1,"uuid":"77ea4ec1-f61c-4839-b2c4-453c0f8e2415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6801","nonce":1533372313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6803","nonce":1533372313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6807","nonce":1533372313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1533372313},{"type":"v1","addr":"172.21.15.181:6805","nonce":1533372313}]},"public_addr":"172.21.15.181:6801/1533372313","cluster_addr":"172.21.15.181:6803/1533372313","heartbeat_back_addr":"172.21.15.181:6807/1533372313","heartbeat_front_addr":"172.21.15.181:6805/1533372313","state":["exists","up"]},{"osd":2,"uuid":"6af3e4b4-81a9-4247-b17e-ab3ca353d09b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6801","nonce":82472683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6803","nonce":82472683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6807","nonce":82472683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":82472683},{"type":"v1","addr":"172.21.15.204:6805","nonce":82472683}]},"public_addr":"172.21.15.204:6801/82472683","cluster_addr":"172.21.15.204:6803/82472683","heartbeat_back_addr":"172.21.15.204:6807/82472683","heartbeat_front_addr":"172.21.15.204:6805/82472683","state":["exists","up"]},{"osd":3,"uuid":"4746f9bf-15b6-4dfb-8e93-8140a669499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6810","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6811","nonce":2741769998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6812","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6813","nonce":2741769998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6816","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6817","nonce":2741769998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6814","nonce":2741769998},{"type":"v1","addr":"172.21.15.57:6815","nonce":2741769998}]},"public_addr":"172.21.15.57:6811/2741769998","cluster_addr":"172.21.15.57:6813/2741769998","heartbeat_back_addr":"172.21.15.57:6817/2741769998","heartbeat_front_addr":"172.21.15.57:6815/2741769998","state":["exists","up"]},{"osd":4,"uuid":"9ed45fbb-6c70-460b-8f77-77f8810b2e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6809","nonce":3730474158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6811","nonce":3730474158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6815","nonce":3730474158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":3730474158},{"type":"v1","addr":"172.21.15.181:6813","nonce":3730474158}]},"public_addr":"172.21.15.181:6809/3730474158","cluster_addr":"172.21.15.181:6811/3730474158","heartbeat_back_addr":"172.21.15.181:6815/3730474158","heartbeat_front_addr":"172.21.15.181:6813/3730474158","state":["exists","up"]},{"osd":5,"uuid":"08fe7507-dac1-4941-b250-aa6352886ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6809","nonce":4265194812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6811","nonce":4265194812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6815","nonce":4265194812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4265194812},{"type":"v1","addr":"172.21.15.204:6813","nonce":4265194812}]},"public_addr":"172.21.15.204:6809/4265194812","cluster_addr":"172.21.15.204:6811/4265194812","heartbeat_back_addr":"172.21.15.204:6815/4265194812","heartbeat_front_addr":"172.21.15.204:6813/4265194812","state":["exists","up"]},{"osd":6,"uuid":"791b2b8c-0c86-49ca-a977-e6cb195ad8ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6818","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6819","nonce":1487516632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6820","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6821","nonce":1487516632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6824","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6825","nonce":1487516632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6822","nonce":1487516632},{"type":"v1","addr":"172.21.15.57:6823","nonce":1487516632}]},"public_addr":"172.21.15.57:6819/1487516632","cluster_addr":"172.21.15.57:6821/1487516632","heartbeat_back_addr":"172.21.15.57:6825/1487516632","heartbeat_front_addr":"172.21.15.57:6823/1487516632","state":["exists","up"]},{"osd":7,"uuid":"45285c14-5770-46e0-8eb8-8f8e46d74882","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6817","nonce":159674330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6819","nonce":159674330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6823","nonce":159674330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":159674330},{"type":"v1","addr":"172.21.15.181:6821","nonce":159674330}]},"public_addr":"172.21.15.181:6817/159674330","cluster_addr":"172.21.15.181:6819/159674330","heartbeat_back_addr":"172.21.15.181:6823/159674330","heartbeat_front_addr":"172.21.15.181:6821/159674330","state":["exists","up"]},{"osd":8,"uuid":"2012481a-b076-4538-a4f4-09c1e17f89f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6817","nonce":2786968053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6819","nonce":2786968053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6823","nonce":2786968053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2786968053},{"type":"v1","addr":"172.21.15.204:6821","nonce":2786968053}]},"public_addr":"172.21.15.204:6817/2786968053","cluster_addr":"172.21.15.204:6819/2786968053","heartbeat_back_addr":"172.21.15.204:6823/2786968053","heartbeat_front_addr":"172.21.15.204:6821/2786968053","state":["exists","up"]},{"osd":9,"uuid":"bd16b9d8-bb6d-4def-8089-8ee0c2fc4b51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6826","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6827","nonce":3989705460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6828","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6829","nonce":3989705460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6832","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6833","nonce":3989705460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.57:6830","nonce":3989705460},{"type":"v1","addr":"172.21.15.57:6831","nonce":3989705460}]},"public_addr":"172.21.15.57:6827/3989705460","cluster_addr":"172.21.15.57:6829/3989705460","heartbeat_back_addr":"172.21.15.57:6833/3989705460","heartbeat_front_addr":"172.21.15.57:6831/3989705460","state":["exists","up"]},{"osd":10,"uuid":"46a48ee5-1571-4aa2-91ed-2f972739a137","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6825","nonce":25222354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6827","nonce":25222354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6831","nonce":25222354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":25222354},{"type":"v1","addr":"172.21.15.181:6829","nonce":25222354}]},"public_addr":"172.21.15.181:6825/25222354","cluster_addr":"172.21.15.181:6827/25222354","heartbeat_back_addr":"172.21.15.181:6831/25222354","heartbeat_front_addr":"172.21.15.181:6829/25222354","state":["exists","up"]},{"osd":11,"uuid":"14d16ce7-b559-4958-9d46-03684f5c25ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6825","nonce":1683140882}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6827","nonce":1683140882}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6831","nonce":1683140882}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1683140882},{"type":"v1","addr":"172.21.15.204:6829","nonce":1683140882}]},"public_addr":"172.21.15.204:6825/1683140882","cluster_addr":"172.21.15.204:6827/1683140882","heartbeat_back_addr":"172.21.15.204:6831/1683140882","heartbeat_front_addr":"172.21.15.204:6829/1683140882","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:12.015552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:57:37.871157+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:03.041152+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:32.548683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:58:59.607133+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:24.476145+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:59:54.385506+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:21.463003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:00:47.146710+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:16.770481+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:01:43.339662+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:02:09.799220+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.57:0/2696288462":"2024-09-13T03:05:55.702194+0000","172.21.15.57:0/752811668":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1354301616":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6800/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1176788560":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/1345986708":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1679473944":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6801/522425633":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/2870728447":"2024-09-13T03:05:55.702194+0000","172.21.15.57:6801/1326814975":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6801/480086157":"2024-09-13T03:05:55.702194+0000","172.21.15.57:0/1256058423":"2024-09-13T02:55:10.466284+0000","172.21.15.57:0/1578458239":"2024-09-13T02:55:39.619488+0000","172.21.15.57:0/4021182045":"2024-09-13T02:55:10.466284+0000","172.21.15.57:6800/480086157":"2024-09-13T03:05:55.702194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:50.717 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 msgr2=0x7f62d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 0x7f62d40a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009230 tx=0x7f62dc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 msgr2=0x7f62f019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e9f0 tx=0x7f62e000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 shutdown_connections 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0109140 0x7f62f019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f62d40a2cf0 0x7f62d40a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f62f01049c0 0x7f62f019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 --2- 172.21.15.57:0/4285829638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62f0104010 0x7f62f019a6b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 >> 172.21.15.57:0/4285829638 conn(0x7f62f00ffa50 msgr2=0x7f62f0100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 shutdown_connections 2024-09-12T03:17:50.718 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:50.717+0000 7f62f6a00640 1 -- 172.21.15.57:0/4285829638 wait complete. 2024-09-12T03:17:51.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:51 smithi057 ceph-mon[30361]: pgmap v373: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:17:51.495 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:51 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/4285829638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:51.533 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:17:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:51 smithi204 ceph-mon[35013]: pgmap v373: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:17:51.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:51 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/4285829638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:51 smithi181 ceph-mon[35320]: pgmap v373: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T03:17:51.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:51 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/4285829638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:51.913 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.532+0000 7f6c33e00640 1 -- 172.21.15.57:0/1269879638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34077040 msgr2=0x7f6c34112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.532+0000 7f6c33e00640 1 --2- 172.21.15.57:0/1269879638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34077040 0x7f6c34112330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009900 tx=0x7f6c1c02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 -- 172.21.15.57:0/1269879638 shutdown_connections 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 --2- 172.21.15.57:0/1269879638 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34112870 0x7f6c34114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 --2- 172.21.15.57:0/1269879638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34077040 0x7f6c34112330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 --2- 172.21.15.57:0/1269879638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34076720 0x7f6c34076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 -- 172.21.15.57:0/1269879638 >> 172.21.15.57:0/1269879638 conn(0x7f6c340ff8d0 msgr2=0x7f6c34101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 -- 172.21.15.57:0/1269879638 shutdown_connections 2024-09-12T03:17:52.541 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.533+0000 7f6c33e00640 1 -- 172.21.15.57:0/1269879638 wait complete. 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 Processor -- start 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 -- start start 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34077040 0x7f6c341a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34112870 0x7f6c3419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f6c34118450 con 0x7f6c34077040 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341182d0 con 0x7f6c34112870 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341185d0 con 0x7f6c34076720 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c2a000640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34077040 0x7f6c341a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c32a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c32a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:32990/0 (socket says 172.21.15.57:32990) 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c32a00640 1 -- 172.21.15.57:0/3239890366 learned_addr learned my addr 172.21.15.57:0/3239890366 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.534+0000 7f6c33400640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34112870 0x7f6c3419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.542 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 -- 172.21.15.57:0/3239890366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34112870 msgr2=0x7f6c3419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34112870 0x7f6c3419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 -- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34077040 msgr2=0x7f6c341a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34077040 0x7f6c341a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 -- 172.21.15.57:0/3239890366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3419ef60 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c32a00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e9f0 tx=0x7f6c2400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400cd80 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24005500 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24010690 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.535+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3419f250 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.536+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c341ab4a0 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.537+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c34077e60 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.539+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f6c240046c0 con 0x7f6c34076720 2024-09-12T03:17:52.543 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.540+0000 7f6c2be00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 0x7f6c180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.540+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9538+0+0 (secure 0 0 0) 0x7f6c240cfbf0 con 0x7f6c34076720 2024-09-12T03:17:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.540+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 0x7f6c180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.541+0000 7f6c2a000640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 0x7f6c180a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c005b50 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:52.544 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.542+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c2409c3b0 con 0x7f6c34076720 2024-09-12T03:17:52.767 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.767+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c3410da00 con 0x7f6c34076720 2024-09-12T03:17:52.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.781+0000 7f6c2be00640 1 -- 172.21.15.57:0/3239890366 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7f6c2409bcb0 con 0x7f6c34076720 2024-09-12T03:17:52.781 INFO:teuthology.orchestra.run.smithi057.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 msgr2=0x7f6c180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 0x7f6c180a5230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c005b50 comp rx=0 tx=0).stop 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 msgr2=0x7f6c341a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e9f0 tx=0x7f6c2400edf0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 shutdown_connections 2024-09-12T03:17:52.784 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c34112870 0x7f6c3419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f6c180a2d70 0x7f6c180a5230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f6c34077040 0x7f6c341a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 --2- 172.21.15.57:0/3239890366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c34076720 0x7f6c341a4640 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 >> 172.21.15.57:0/3239890366 conn(0x7f6c340ff8d0 msgr2=0x7f6c34101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 shutdown_connections 2024-09-12T03:17:52.785 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:52.784+0000 7f6c33e00640 1 -- 172.21.15.57:0/3239890366 wait complete. 2024-09-12T03:17:53.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a[30357]: 2024-09-12T03:17:52.769+0000 7fa381800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:53.120 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:17:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a[71252]: starting mds.a at 2024-09-12T03:17:53.144 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:53.144 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:53.144 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-mds[65439]: -- 172.21.15.204:0/119648851 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56496e60d860 con 0x56496e5ea800 2024-09-12T03:17:53.144 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: starting mds.c at 2024-09-12T03:17:53.144 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-mds[65138]: -- 172.21.15.204:0/521158700 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563470b4d860 con 0x563470b2b000 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: starting mds.f at 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-mds[65742]: -- 172.21.15.204:0/1198331924 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f754245860 con 0x55f754223000 2024-09-12T03:17:53.145 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:17:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: starting mds.i at 2024-09-12T03:17:53.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:53.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:53.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-mds[65419]: -- 172.21.15.181:0/384593533 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618167cd860 con 0x5618167aa800 2024-09-12T03:17:53.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: starting mds.b at 2024-09-12T03:17:53.204 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: ignoring --setuser ceph since I am not root 2024-09-12T03:17:53.204 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: ignoring --setgroup ceph since I am not root 2024-09-12T03:17:53.204 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-mds[65107]: -- 172.21.15.181:0/2814206981 <== mon.2 v2:172.21.15.181:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c4baa7c20 con 0x557c4ba85000 2024-09-12T03:17:53.204 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: starting mds.h at 2024-09-12T03:17:53.204 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:17:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: 2024-09-12T03:17:52.820+0000 7f9b02c72640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: pgmap v374: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3239890366' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:17:53.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:53 smithi057 ceph-mon[30361]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:17:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: pgmap v374: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T03:17:53.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3239890366' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:17:53.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:53 smithi204 ceph-mon[35013]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: pgmap v374: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3239890366' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T03:17:53.703 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:53 smithi181 ceph-mon[35320]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T03:17:53.817 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:17:54.197 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:54.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 -- 172.21.15.57:0/3554525046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 msgr2=0x7f78f8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 --2- 172.21.15.57:0/3554525046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f8076b00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009900 tx=0x7f78e802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 -- 172.21.15.57:0/3554525046 shutdown_connections 2024-09-12T03:17:54.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 --2- 172.21.15.57:0/3554525046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8112a30 0x7f78f8114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.777 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 --2- 172.21.15.57:0/3554525046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8077040 0x7f78f81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 --2- 172.21.15.57:0/3554525046 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f8076b00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 -- 172.21.15.57:0/3554525046 >> 172.21.15.57:0/3554525046 conn(0x7f78f80ffa90 msgr2=0x7f78f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 -- 172.21.15.57:0/3554525046 shutdown_connections 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.751+0000 7f78fe800640 1 -- 172.21.15.57:0/3554525046 wait complete. 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 Processor -- start 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 -- start start 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 0x7f78f81a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8112a30 0x7f78f819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f78f81185c0 con 0x7f78f8076720 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8118440 con 0x7f78f8077040 2024-09-12T03:17:54.778 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.752+0000 7f78fe800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8118740 con 0x7f78f8112a30 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78f7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 0x7f78f81a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78f7400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58062/0 (socket says 172.21.15.57:58062) 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78f7400640 1 -- 172.21.15.57:0/1215221132 learned_addr learned my addr 172.21.15.57:0/1215221132 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78f7e00640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8112a30 0x7f78f819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 -- 172.21.15.57:0/1215221132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8112a30 msgr2=0x7f78f819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8112a30 0x7f78f819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 -- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 msgr2=0x7f78f81a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 -- 172.21.15.57:0/1215221132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f819efe0 con 0x7f78f8077040 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78f7400640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.753+0000 7f78efe00640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 0x7f78f81a4c30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e9f0 tx=0x7f78e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.754+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e000ccc0 con 0x7f78f8077040 2024-09-12T03:17:54.779 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.754+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e0005020 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.754+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f819f2d0 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.754+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e00049e0 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.754+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f81ab5c0 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.755+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8076b00 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.758+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f78e0005560 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.759+0000 7f78f4c00640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 0x7f78dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.759+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9724+0+0 (secure 0 0 0) 0x7f78e00cf970 con 0x7f78f8077040 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.759+0000 7f78f7400640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 0x7f78dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.760+0000 7f78f7400640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 0x7f78dc0a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e8031550 tx=0x7f78e8005a20 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:54.780 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.760+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78e00051d0 con 0x7f78f8077040 2024-09-12T03:17:54.982 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:54.983+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f78f81ab8b0 con 0x7f78f8077040 2024-09-12T03:17:54.982 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6837/146855285,v1:172.21.15.181:6839/146855285] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.181:6836/1896140480,v1:172.21.15.181:6838/1896140480] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6834/2092853492,v1:172.21.15.204:6835/2092853492] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6836/2735986572,v1:172.21.15.204:6838/2735986572] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.204:6837/1873016864,v1:172.21.15.204:6839/1873016864] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: mds.? [v2:172.21.15.57:6840/2706379377,v1:172.21.15.57:6841/2706379377] up:boot 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:17:54.983 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:54 smithi057 ceph-mon[30361]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:17:55.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6837/146855285,v1:172.21.15.181:6839/146855285] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.181:6836/1896140480,v1:172.21.15.181:6838/1896140480] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6834/2092853492,v1:172.21.15.204:6835/2092853492] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6836/2735986572,v1:172.21.15.204:6838/2735986572] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.204:6837/1873016864,v1:172.21.15.204:6839/1873016864] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: mds.? [v2:172.21.15.57:6840/2706379377,v1:172.21.15.57:6841/2706379377] up:boot 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:17:55.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:17:55.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:54 smithi204 ceph-mon[35013]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6837/146855285,v1:172.21.15.181:6839/146855285] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.181:6836/1896140480,v1:172.21.15.181:6838/1896140480] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6834/2092853492,v1:172.21.15.204:6835/2092853492] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6836/2735986572,v1:172.21.15.204:6838/2735986572] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.204:6837/1873016864,v1:172.21.15.204:6839/1873016864] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: mds.? [v2:172.21.15.57:6840/2706379377,v1:172.21.15.57:6841/2706379377] up:boot 2024-09-12T03:17:55.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:17:55.204 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:54 smithi181 ceph-mon[35320]: from='mgr.15126 172.21.15.57:0/2462826582' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:17:55.798 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.799+0000 7f78f4c00640 1 -- 172.21.15.57:0/1215221132 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) ==== 104+0+0 (secure 0 0 0) 0x7f78e009c110 con 0x7f78f8077040 2024-09-12T03:17:55.801 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 msgr2=0x7f78dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 0x7f78dc0a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e8031550 tx=0x7f78e8005a20 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 msgr2=0x7f78f81a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 0x7f78f81a4c30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f78e000e9f0 tx=0x7f78e000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 shutdown_connections 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8112a30 0x7f78f819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78f8077040 0x7f78f81a4c30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f78dc0a2cf0 0x7f78dc0a51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 --2- 172.21.15.57:0/1215221132 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f78f8076720 0x7f78f81a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 >> 172.21.15.57:0/1215221132 conn(0x7f78f80ffa90 msgr2=0x7f78f8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 shutdown_connections 2024-09-12T03:17:55.802 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:55.802+0000 7f78fe800640 1 -- 172.21.15.57:0/1215221132 wait complete. 2024-09-12T03:17:56.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:55 smithi057 ceph-mon[30361]: pgmap v376: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:17:56.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:55 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1215221132' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.026 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:55 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:55 smithi181 ceph-mon[35320]: pgmap v376: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:17:56.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:55 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1215221132' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.105 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:55 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:55 smithi204 ceph-mon[35013]: pgmap v376: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-12T03:17:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:55 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1215221132' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:55 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:17:56.669 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:17:57.050 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:56 smithi057 ceph-mon[30361]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:56 smithi057 ceph-mon[30361]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:56 smithi057 ceph-mon[30361]: Cluster is now healthy 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:56 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:56 smithi057 ceph-mon[30361]: fsmap 12 up:standby 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:56 smithi181 ceph-mon[35320]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:56 smithi181 ceph-mon[35320]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:56 smithi181 ceph-mon[35320]: Cluster is now healthy 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:56 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:17:57.120 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:56 smithi181 ceph-mon[35320]: fsmap 12 up:standby 2024-09-12T03:17:57.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:56 smithi204 ceph-mon[35013]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:17:57.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:56 smithi204 ceph-mon[35013]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:17:57.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:56 smithi204 ceph-mon[35013]: Cluster is now healthy 2024-09-12T03:17:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:56 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:17:57.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:56 smithi204 ceph-mon[35013]: fsmap 12 up:standby 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 -- 172.21.15.57:0/1271063222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 msgr2=0x7f8e04108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 --2- 172.21.15.57:0/1271063222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e04108a10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009860 tx=0x7f8df002a340 comp rx=0 tx=0).stop 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 -- 172.21.15.57:0/1271063222 shutdown_connections 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 --2- 172.21.15.57:0/1271063222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e04110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 --2- 172.21.15.57:0/1271063222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e04108a10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 --2- 172.21.15.57:0/1271063222 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 -- 172.21.15.57:0/1271063222 >> 172.21.15.57:0/1271063222 conn(0x7f8e040ffa50 msgr2=0x7f8e04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 -- 172.21.15.57:0/1271063222 shutdown_connections 2024-09-12T03:17:57.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.650+0000 7f8e0c400640 1 -- 172.21.15.57:0/1271063222 wait complete. 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 Processor -- start 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 -- start start 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e0419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e0419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04114140 con 0x7f8e04104010 2024-09-12T03:17:57.653 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f8e04113fc0 con 0x7f8e04109140 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.651+0000 7f8e0c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e041142c0 con 0x7f8e041049c0 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e09a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e0419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.57:38154/0 (socket says 172.21.15.57:38154) 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 -- 172.21.15.57:0/3508906097 learned_addr learned my addr 172.21.15.57:0/3508906097 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e09a00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e0419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:58080/0 (socket says 172.21.15.57:58080) 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e09000640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e0419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 -- 172.21.15.57:0/3508906097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 msgr2=0x7f8e0419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e0419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 -- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 msgr2=0x7f8e0419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e0419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.652+0000 7f8e0a400640 1 -- 172.21.15.57:0/3508906097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0419f610 con 0x7f8e04109140 2024-09-12T03:17:57.654 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e0a400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00efd0 tx=0x7f8dfc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc003b70 con 0x7f8e04109140 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0419f900 con 0x7f8e04109140 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc003d10 con 0x7f8e04109140 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0108d0 con 0x7f8e04109140 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.653+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e041a71e0 con 0x7f8e04109140 2024-09-12T03:17:57.655 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.655+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e041043f0 con 0x7f8e04109140 2024-09-12T03:17:57.656 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.656+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f8dfc004020 con 0x7f8e04109140 2024-09-12T03:17:57.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.657+0000 7f8e02000640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 0x7f8dec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:57.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.658+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9724+0+0 (secure 0 0 0) 0x7f8dfc0cfc50 con 0x7f8e04109140 2024-09-12T03:17:57.658 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.658+0000 7f8e09a00640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 0x7f8dec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:57.659 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.658+0000 7f8e09a00640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 0x7f8dec0a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df8009260 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:57.661 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.661+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8dfc09c360 con 0x7f8e04109140 2024-09-12T03:17:57.849 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:57.848+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8e0410c3e0 con 0x7f8e04109140 2024-09-12T03:17:58.057 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:57 smithi057 ceph-mon[30361]: pgmap v377: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-12T03:17:58.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:57 smithi204 ceph-mon[35013]: pgmap v377: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-12T03:17:58.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:57 smithi181 ceph-mon[35320]: pgmap v377: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-12T03:17:58.815 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.814+0000 7f8e02000640 1 -- 172.21.15.57:0/3508906097 <== mon.2 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f8dfc09bc60 con 0x7f8e04109140 2024-09-12T03:17:58.815 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.816+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 msgr2=0x7f8dec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.816+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 0x7f8dec0a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df8009260 comp rx=0 tx=0).stop 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.816+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 msgr2=0x7f8e0419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.816+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00efd0 tx=0x7f8dfc00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 shutdown_connections 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e04109140 0x7f8e0419ef30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.817 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e041049c0 0x7f8e0419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f8dec0a2d70 0x7f8dec0a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 --2- 172.21.15.57:0/3508906097 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f8e04104010 0x7f8e0419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 >> 172.21.15.57:0/3508906097 conn(0x7f8e040ffa50 msgr2=0x7f8e04100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 shutdown_connections 2024-09-12T03:17:58.818 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:17:58.817+0000 7f8e0c400640 1 -- 172.21.15.57:0/3508906097 wait complete. 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:58 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/3508906097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:58 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:58 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/3508906097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:58 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:58 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/3508906097' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:58 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:17:59.658 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:17:59.869 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:59 smithi057 ceph-mon[30361]: pgmap v378: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-12T03:17:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:59 smithi057 ceph-mon[30361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:17:59.870 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:17:59 smithi057 ceph-mon[30361]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:18:00.042 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:59 smithi204 ceph-mon[35013]: pgmap v378: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-12T03:18:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:59 smithi204 ceph-mon[35013]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:00.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:17:59 smithi204 ceph-mon[35013]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:18:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:59 smithi181 ceph-mon[35320]: pgmap v378: 129 pgs: 129 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-12T03:18:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:59 smithi181 ceph-mon[35320]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:00.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:17:59 smithi181 ceph-mon[35320]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T03:18:00.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 -- 172.21.15.57:0/3362516813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17781047e0 msgr2=0x7f1778108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:00.640 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 --2- 172.21.15.57:0/3362516813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17781047e0 0x7f1778108830 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1760009900 tx=0x7f176002a410 comp rx=0 tx=0).stop 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 -- 172.21.15.57:0/3362516813 shutdown_connections 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 --2- 172.21.15.57:0/3362516813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1778108f60 0x7f1778110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 --2- 172.21.15.57:0/3362516813 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f17781047e0 0x7f1778108830 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 --2- 172.21.15.57:0/3362516813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f1778104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 -- 172.21.15.57:0/3362516813 >> 172.21.15.57:0/3362516813 conn(0x7f17780ff8d0 msgr2=0x7f1778101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 -- 172.21.15.57:0/3362516813 shutdown_connections 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.638+0000 7f1777200640 1 -- 172.21.15.57:0/3362516813 wait complete. 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.639+0000 7f1777200640 1 Processor -- start 2024-09-12T03:18:00.641 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.639+0000 7f1777200640 1 -- start start 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.639+0000 7f1777200640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f177819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.639+0000 7f1777200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 0x7f177819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.639+0000 7f1777200640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1777200640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7f1778114010 con 0x7f1778108f60 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1777200640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7f1778113e90 con 0x7f1778103e30 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1777200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1778114190 con 0x7f17781047e0 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f176d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 0x7f177819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f176d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 0x7f177819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.57:55008/0 (socket says 172.21.15.57:55008) 2024-09-12T03:18:00.642 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f176d400640 1 -- 172.21.15.57:0/1253795534 learned_addr learned my addr 172.21.15.57:0/1253795534 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:18:00.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52098/0 (socket says 172.21.15.57:52098) 2024-09-12T03:18:00.643 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1775e00640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f177819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 -- 172.21.15.57:0/1253795534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 msgr2=0x7f177819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 0x7f177819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 -- 172.21.15.57:0/1253795534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 msgr2=0x7f177819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f177819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:00.644 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1776800640 1 -- 172.21.15.57:0/1253795534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177819f420 con 0x7f1778108f60 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.640+0000 7f1775e00640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f177819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f1776800640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f177000e940 tx=0x7f177000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770004d20 con 0x7f1778108f60 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1770005020 con 0x7f1778108f60 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177819f710 con 0x7f1778108f60 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700116b0 con 0x7f1778108f60 2024-09-12T03:18:00.645 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.641+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17781a6ff0 con 0x7f1778108f60 2024-09-12T03:18:00.646 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.642+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1778104210 con 0x7f1778108f60 2024-09-12T03:18:00.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.644+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7f17700054d0 con 0x7f1778108f60 2024-09-12T03:18:00.647 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.646+0000 7f176f400640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 0x7f175c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:00.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.646+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9170+0+0 (secure 0 0 0) 0x7f17700cfa60 con 0x7f1778108f60 2024-09-12T03:18:00.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.646+0000 7f1775e00640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 0x7f175c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:00.648 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.647+0000 7f1775e00640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 0x7f175c0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f176800bfd0 tx=0x7f176800b390 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:00.651 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.650+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1770016030 con 0x7f1778108f60 2024-09-12T03:18:00.836 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:00.835+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f177802d7c0 con 0x7f1778108f60 2024-09-12T03:18:01.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:18:00.948+0000 7fd85e000640 -1 client.25201: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T03:18:01.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:18:00.950+0000 7fd85e000640 -1 client.25201: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T03:18:01.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Exception in thread Thread-17: 2024-09-12T03:18:01.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: Traceback (most recent call last): 2024-09-12T03:18:01.120 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.run() 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.function(*self.args, **self.kwargs) 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: self.prune_snapshots(sched) 2024-09-12T03:18:01.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-12T03:18:01.819 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.819+0000 7f176f400640 1 -- 172.21.15.57:0/1253795534 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7f177009bd40 con 0x7f1778108f60 2024-09-12T03:18:01.820 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data' removed 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 msgr2=0x7f175c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 0x7f175c0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f176800bfd0 tx=0x7f176800b390 comp rx=0 tx=0).stop 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 msgr2=0x7f177819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f177000e940 tx=0x7f177000ee00 comp rx=0 tx=0).stop 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 shutdown_connections 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7f175c0a2cf0 0x7f175c0a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7f1778108f60 0x7f177819ed40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.824 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17781047e0 0x7f177819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 --2- 172.21.15.57:0/1253795534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1778103e30 0x7f177819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 >> 172.21.15.57:0/1253795534 conn(0x7f17780ff8d0 msgr2=0x7f1778100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 shutdown_connections 2024-09-12T03:18:01.825 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:01.823+0000 7f1777200640 1 -- 172.21.15.57:0/1253795534 wait complete. 2024-09-12T03:18:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:01 smithi057 ceph-mon[30361]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:02.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:01 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:01 smithi204 ceph-mon[35013]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:02.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:01 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:01 smithi181 ceph-mon[35320]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:02.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:01 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:02.587 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-12T03:18:02.967 INFO:teuthology.orchestra.run.smithi057.stderr:Inferring config /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:03.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:02 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:03.109 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:02 smithi057 ceph-mon[30361]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:18:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:02 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:03.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:02 smithi204 ceph-mon[35013]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:18:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:02 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/1253795534' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:03.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:02 smithi181 ceph-mon[35320]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T03:18:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 -- 172.21.15.57:0/1857625716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda641049c0 msgr2=0x7fda64108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 --2- 172.21.15.57:0/1857625716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda641049c0 0x7fda64108a10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda50009860 tx=0x7fda5002a340 comp rx=0 tx=0).stop 2024-09-12T03:18:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 -- 172.21.15.57:0/1857625716 shutdown_connections 2024-09-12T03:18:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 --2- 172.21.15.57:0/1857625716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda64109140 0x7fda64110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.601 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 --2- 172.21.15.57:0/1857625716 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda641049c0 0x7fda64108a10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 --2- 172.21.15.57:0/1857625716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64104010 0x7fda641043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 -- 172.21.15.57:0/1857625716 >> 172.21.15.57:0/1857625716 conn(0x7fda640ffa50 msgr2=0x7fda64101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 -- 172.21.15.57:0/1857625716 shutdown_connections 2024-09-12T03:18:03.602 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.600+0000 7fda68e00640 1 -- 172.21.15.57:0/1857625716 wait complete. 2024-09-12T03:18:03.603 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.601+0000 7fda68e00640 1 Processor -- start 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.601+0000 7fda68e00640 1 -- start start 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 0x7fda6419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 0x7fda6419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 -- --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_getmap magic: 0 -- 0x7fda64114140 con 0x7fda64104010 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 -- 0x7fda64113fc0 con 0x7fda64109140 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda68e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda641142c0 con 0x7fda641049c0 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda61e00640 1 --2- >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.57:3300/0 says I am v2:172.21.15.57:52120/0 (socket says 172.21.15.57:52120) 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda61e00640 1 -- 172.21.15.57:0/2362030306 learned_addr learned my addr 172.21.15.57:0/2362030306 (peer_addr_for_me v2:172.21.15.57:0/0) 2024-09-12T03:18:03.604 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda61400640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 0x7fda6419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.602+0000 7fda62800640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 0x7fda6419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 -- 172.21.15.57:0/2362030306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 msgr2=0x7fda6419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 0x7fda6419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 -- 172.21.15.57:0/2362030306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 msgr2=0x7fda6419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 0x7fda6419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 -- 172.21.15.57:0/2362030306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6419f610 con 0x7fda64104010 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61400640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 0x7fda6419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda62800640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 0x7fda6419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda61e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda5400ea60 tx=0x7fda5400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5400cde0 con 0x7fda64104010 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda540035f0 con 0x7fda64104010 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda54010670 con 0x7fda64104010 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6419f900 con 0x7fda64104010 2024-09-12T03:18:03.605 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.603+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda641a71e0 con 0x7fda64104010 2024-09-12T03:18:03.606 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.604+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda641043f0 con 0x7fda64104010 2024-09-12T03:18:03.607 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.606+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 4 ==== mgrmap(e 27) ==== 137964+0+0 (secure 0 0 0) 0x7fda540037a0 con 0x7fda64104010 2024-09-12T03:18:03.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.608+0000 7fda5aa00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 0x7fda480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.608+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8582+0+0 (secure 0 0 0) 0x7fda540cfa10 con 0x7fda64104010 2024-09-12T03:18:03.608 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.608+0000 7fda61400640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 0x7fda480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.609 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.609+0000 7fda61400640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 0x7fda480a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda5002a930 tx=0x7fda50005a90 comp rx=0 tx=0).ready entity=mgr.15126 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.611 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.610+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda5401f340 con 0x7fda64104010 2024-09-12T03:18:03.797 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.797+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 --> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda6419b9c0 con 0x7fda64104010 2024-09-12T03:18:03.835 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.834+0000 7fda5aa00640 1 -- 172.21.15.57:0/2362030306 <== mon.0 v2:172.21.15.57:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v114) ==== 178+0+0 (secure 0 0 0) 0x7fda5409c620 con 0x7fda64104010 2024-09-12T03:18:03.835 INFO:teuthology.orchestra.run.smithi057.stderr:pool 'cephfs_data_ec' removed 2024-09-12T03:18:03.839 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.838+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 msgr2=0x7fda480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.838+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 0x7fda480a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fda5002a930 tx=0x7fda50005a90 comp rx=0 tx=0).stop 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.838+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 msgr2=0x7fda6419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.838+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fda5400ea60 tx=0x7fda5400ef20 comp rx=0 tx=0).stop 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 shutdown_connections 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda64109140 0x7fda6419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda641049c0 0x7fda6419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.840 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:6800/260348644,v1:172.21.15.57:6801/260348644] conn(0x7fda480a2cf0 0x7fda480a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 --2- 172.21.15.57:0/2362030306 >> [v2:172.21.15.57:3300/0,v1:172.21.15.57:6789/0] conn(0x7fda64104010 0x7fda6419a660 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.839+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 >> 172.21.15.57:0/2362030306 conn(0x7fda640ffa50 msgr2=0x7fda64100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.841+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 shutdown_connections 2024-09-12T03:18:03.841 INFO:teuthology.orchestra.run.smithi057.stderr:2024-09-12T03:18:03.841+0000 7fda68e00640 1 -- 172.21.15.57:0/2362030306 wait complete. 2024-09-12T03:18:04.119 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:03 smithi057 ceph-mon[30361]: pgmap v382: 33 pgs: 33 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:04.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:03 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:04.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:03 smithi204 ceph-mon[35013]: pgmap v382: 33 pgs: 33 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:04.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:03 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:03 smithi181 ceph-mon[35320]: pgmap v382: 33 pgs: 33 active+clean; 10 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:18:04.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:03 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:18:04.836 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:04.894 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:04.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:18:04.954 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:18:04.954 DEBUG:teuthology.orchestra.run.smithi057:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:18:04.996 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:18:05.013 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:18:05.032 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:18:05.032 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:18:05.032 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:18:05.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:04 smithi057 ceph-mon[30361]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:05.084 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:04 smithi057 ceph-mon[30361]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:18:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:04 smithi204 ceph-mon[35013]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:05.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:04 smithi204 ceph-mon[35013]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:18:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:04 smithi181 ceph-mon[35320]: from='client.? 172.21.15.57:0/2362030306' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:18:05.203 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:04 smithi181 ceph-mon[35320]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T03:18:05.370 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:05 smithi057 systemd[1]: Stopping Ceph mon.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a[30357]: 2024-09-12T03:18:05.781+0000 7fa388600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a[30357]: 2024-09-12T03:18:05.781+0000 7fa388600640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:18:06.120 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:05 smithi057 podman[92740]: 2024-09-12 03:18:05.987585045 +0000 UTC m=+0.592049333 container died 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:18:06.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:06 smithi057 podman[92740]: 2024-09-12 03:18:06.430524927 +0000 UTC m=+1.034989205 container cleanup 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:18:06.620 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:06 smithi057 bash[92740]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:18:06.911 INFO:journalctl@ceph.mon.a.smithi057.stdout:Sep 12 03:18:06 smithi057 podman[92754]: 2024-09-12 03:18:06.657219869 +0000 UTC m=+0.662186282 container remove 6aafe956a4107dffe314fd447ea799845bfb6d3595bae0c0c518cd8a0ee7de1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:18:06.939 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:18:07.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:07.021 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:18:07.021 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:18:07.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:18:07.350 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 systemd[1]: Stopping Ceph mon.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:07.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-b[35316]: 2024-09-12T03:18:07.350+0000 7fefe85cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:07.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-b[35316]: 2024-09-12T03:18:07.350+0000 7fefe85cb640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:18:07.675 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 podman[71290]: 2024-09-12 03:18:07.391407556 +0000 UTC m=+0.172838127 container died 2f6e4b2e383d1d023ac5ca516b3722b72fae68e3dd4ae0ba8ac56b601d1e986e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 podman[71290]: 2024-09-12 03:18:07.775702014 +0000 UTC m=+0.557132575 container cleanup 2f6e4b2e383d1d023ac5ca516b3722b72fae68e3dd4ae0ba8ac56b601d1e986e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:18:07.953 INFO:journalctl@ceph.mon.b.smithi181.stdout:Sep 12 03:18:07 smithi181 bash[71290]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:18:08.245 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:18:08.294 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:08.294 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:18:08.294 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:18:08.294 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:18:08.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:08 smithi204 systemd[1]: Stopping Ceph mon.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:09.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c[35009]: 2024-09-12T03:18:08.895+0000 7fb1fed6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:09.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c[35009]: 2024-09-12T03:18:08.895+0000 7fb1fed6f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:18:09.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:09 smithi204 podman[71601]: 2024-09-12 03:18:09.050005082 +0000 UTC m=+0.561262565 container died 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:18:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:09 smithi204 podman[71601]: 2024-09-12 03:18:09.430348304 +0000 UTC m=+0.941605775 container cleanup 312615cfb88f8775e7f2d0e7c8be11b3c19b88106295b4692b8c5661383570f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:18:09.643 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 12 03:18:09 smithi204 bash[71601]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:18:09.900 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:18:09.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:09.947 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:18:09.947 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:18:09.947 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:18:10.370 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:10 smithi057 systemd[1]: Stopping Ceph mgr.x for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:10.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:18:10.371+0000 7fd930400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:10.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x[30601]: 2024-09-12T03:18:10.371+0000 7fd930400640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T03:18:10.870 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:10 smithi057 podman[92858]: 2024-09-12 03:18:10.485669018 +0000 UTC m=+0.240686872 container died c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:18:11.619 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:11 smithi057 podman[92858]: 2024-09-12 03:18:11.494697968 +0000 UTC m=+1.249715821 container cleanup c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:18:11.620 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:11 smithi057 bash[92858]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:18:12.121 INFO:journalctl@ceph.mgr.x.smithi057.stdout:Sep 12 03:18:11 smithi057 podman[92870]: 2024-09-12 03:18:11.780180954 +0000 UTC m=+1.279297196 container remove c1d562973cb08e7a78d5c4de37b24781187355f76ec096544d49d7e8ecf113d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:18:12.136 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:18:12.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:12.209 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:18:12.209 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:18:12.210 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:18:12.542 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:12 smithi181 systemd[1]: Stopping Ceph mgr.y for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:12.856 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:12 smithi181 podman[71413]: 2024-09-12 03:18:12.543195153 +0000 UTC m=+0.140892056 container died 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:18:13.203 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:12 smithi181 podman[71413]: 2024-09-12 03:18:12.964549704 +0000 UTC m=+0.562246603 container cleanup 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:18:13.203 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:12 smithi181 bash[71413]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:18:13.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:18:13.761 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:13 smithi181 podman[71426]: 2024-09-12 03:18:13.508082229 +0000 UTC m=+0.960118036 container remove 51dc700a7d52d06ec01513b3e4a2d51ab80000c40153de6f685c4add6d1bd363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:13.761 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:13 smithi181 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-12T03:18:13.761 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:13 smithi181 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-12T03:18:13.761 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:13 smithi181 systemd[1]: Stopped Ceph mgr.y for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:18:13.761 INFO:journalctl@ceph.mgr.y.smithi181.stdout:Sep 12 03:18:13 smithi181 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.y.service: Consumed 11.553s CPU time. 2024-09-12T03:18:13.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:13.774 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:18:13.774 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:18:13.774 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:18:14.143 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:18:13 smithi204 systemd[1]: Stopping Ceph mgr.z for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:14.643 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:18:14 smithi204 podman[71721]: 2024-09-12 03:18:14.391870227 +0000 UTC m=+0.426027381 container died 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T03:18:15.054 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:18:14 smithi204 podman[71721]: 2024-09-12 03:18:14.803334082 +0000 UTC m=+0.837491232 container cleanup 390ff4fc4855b11756dc8f32d99f5fd19788f4227ce10eff1b49f732c6a7bc83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:18:15.055 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 12 03:18:14 smithi204 bash[71721]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:18:15.193 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:18:15.228 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:15.228 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:18:15.228 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:18:15.228 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:18:15.621 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:15 smithi057 systemd[1]: Stopping Ceph osd.0 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:16.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T03:18:15.755+0000 7f7d36400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:16.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T03:18:15.755+0000 7f7d36400640 -1 osd.0 114 *** Got signal Terminated *** 2024-09-12T03:18:16.120 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0[44249]: 2024-09-12T03:18:15.755+0000 7f7d36400640 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:21.693 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:21 smithi057 podman[92975]: 2024-09-12 03:18:21.403499979 +0000 UTC m=+5.874967813 container died 8840511dad4da9cb02b10eea6a0058c102f63dc458d57f0d1489b0832445e8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:18:22.110 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:21 smithi057 podman[92975]: 2024-09-12 03:18:21.83357355 +0000 UTC m=+6.305041382 container cleanup 8840511dad4da9cb02b10eea6a0058c102f63dc458d57f0d1489b0832445e8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:18:22.110 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:21 smithi057 bash[92975]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:18:22.369 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:22 smithi057 podman[92994]: 2024-09-12 03:18:22.110254134 +0000 UTC m=+0.702868499 container remove 8840511dad4da9cb02b10eea6a0058c102f63dc458d57f0d1489b0832445e8a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:18:22.947 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:22 smithi057 podman[93081]: 2024-09-12 03:18:22.56311065 +0000 UTC m=+0.027851307 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:22.947 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:22 smithi057 podman[93081]: 2024-09-12 03:18:22.694570997 +0000 UTC m=+0.159311641 container create f5eda883aa751ed338dcf42276cf799b8f87bdc0d7f8c9a7c956056190cfc9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:23.370 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:22 smithi057 podman[93081]: 2024-09-12 03:18:22.947482378 +0000 UTC m=+0.412223016 container init f5eda883aa751ed338dcf42276cf799b8f87bdc0d7f8c9a7c956056190cfc9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:23.370 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:22 smithi057 podman[93081]: 2024-09-12 03:18:22.954551537 +0000 UTC m=+0.419292184 container start f5eda883aa751ed338dcf42276cf799b8f87bdc0d7f8c9a7c956056190cfc9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:18:23.370 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:23 smithi057 podman[93081]: 2024-09-12 03:18:23.047253634 +0000 UTC m=+0.511994272 container attach f5eda883aa751ed338dcf42276cf799b8f87bdc0d7f8c9a7c956056190cfc9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:23.710 INFO:journalctl@ceph.osd.0.smithi057.stdout:Sep 12 03:18:23 smithi057 podman[93104]: 2024-09-12 03:18:23.414320863 +0000 UTC m=+0.026985347 container died f5eda883aa751ed338dcf42276cf799b8f87bdc0d7f8c9a7c956056190cfc9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:18:24.127 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:18:24.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:24.205 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:18:24.205 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:18:24.205 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:18:24.519 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:24 smithi181 systemd[1]: Stopping Ceph osd.1 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:24.953 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T03:18:24.519+0000 7fc4f8ed4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:24.953 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T03:18:24.519+0000 7fc4f8ed4640 -1 osd.1 114 *** Got signal Terminated *** 2024-09-12T03:18:24.953 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1[39786]: 2024-09-12T03:18:24.519+0000 7fc4f8ed4640 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:30.521 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:30 smithi181 podman[71540]: 2024-09-12 03:18:30.232621234 +0000 UTC m=+5.828350664 container died efb7545e49213fce0aa3f0f293505d496d7d5eef8dbad4cc3efe64bf61f30c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:18:30.837 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:30 smithi181 podman[71540]: 2024-09-12 03:18:30.619917728 +0000 UTC m=+6.215647154 container cleanup efb7545e49213fce0aa3f0f293505d496d7d5eef8dbad4cc3efe64bf61f30c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:18:30.837 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:30 smithi181 bash[71540]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:18:31.203 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:30 smithi181 podman[71555]: 2024-09-12 03:18:30.888301149 +0000 UTC m=+0.657138038 container remove efb7545e49213fce0aa3f0f293505d496d7d5eef8dbad4cc3efe64bf61f30c0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:18:31.549 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.136846322 +0000 UTC m=+0.016133033 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:31.550 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.28089639 +0000 UTC m=+0.160183092 container create 216784c816bb1fc548876ffbdf4e9e9a25e5f4840545d36e2a23009c364c2494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T03:18:31.953 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.550643612 +0000 UTC m=+0.429930317 container init 216784c816bb1fc548876ffbdf4e9e9a25e5f4840545d36e2a23009c364c2494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:32.323 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.554377302 +0000 UTC m=+0.433664009 container start 216784c816bb1fc548876ffbdf4e9e9a25e5f4840545d36e2a23009c364c2494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:18:32.324 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.659361487 +0000 UTC m=+0.538648207 container attach 216784c816bb1fc548876ffbdf4e9e9a25e5f4840545d36e2a23009c364c2494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:18:32.324 INFO:journalctl@ceph.osd.1.smithi181.stdout:Sep 12 03:18:31 smithi181 podman[71658]: 2024-09-12 03:18:31.747520496 +0000 UTC m=+0.626807221 container died 216784c816bb1fc548876ffbdf4e9e9a25e5f4840545d36e2a23009c364c2494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:18:32.390 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:18:32.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:32.431 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:18:32.431 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:18:32.431 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:18:32.750 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:32 smithi204 systemd[1]: Stopping Ceph osd.2 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:33.021 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T03:18:32.749+0000 7f760cc4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:33.021 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T03:18:32.749+0000 7f760cc4c640 -1 osd.2 114 *** Got signal Terminated *** 2024-09-12T03:18:33.021 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2[39967]: 2024-09-12T03:18:32.749+0000 7f760cc4c640 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:38.709 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:38 smithi204 podman[71847]: 2024-09-12 03:18:38.407373129 +0000 UTC m=+5.779413803 container died 595de2cf67ea401c159a9b0d141575c140c4115e329e0a2902c5a10a5487ab76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:39.085 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:38 smithi204 podman[71847]: 2024-09-12 03:18:38.800036714 +0000 UTC m=+6.172077395 container cleanup 595de2cf67ea401c159a9b0d141575c140c4115e329e0a2902c5a10a5487ab76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:18:39.085 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:38 smithi204 bash[71847]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:18:39.393 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71873]: 2024-09-12 03:18:39.085541684 +0000 UTC m=+0.672446393 container remove 595de2cf67ea401c159a9b0d141575c140c4115e329e0a2902c5a10a5487ab76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:18:39.689 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.310046415 +0000 UTC m=+0.015945196 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:39.689 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.444968559 +0000 UTC m=+0.150867339 container create 766b7fff9a282e74f2cf9880744a74d815b909dc2eb1ce8403ee26d78f38a637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:40.143 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.689726353 +0000 UTC m=+0.395625154 container init 766b7fff9a282e74f2cf9880744a74d815b909dc2eb1ce8403ee26d78f38a637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:18:40.144 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.693439082 +0000 UTC m=+0.399337865 container start 766b7fff9a282e74f2cf9880744a74d815b909dc2eb1ce8403ee26d78f38a637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:18:40.144 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.789637737 +0000 UTC m=+0.495536518 container attach 766b7fff9a282e74f2cf9880744a74d815b909dc2eb1ce8403ee26d78f38a637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:18:40.144 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 12 03:18:39 smithi204 podman[71963]: 2024-09-12 03:18:39.882981108 +0000 UTC m=+0.588879889 container died 766b7fff9a282e74f2cf9880744a74d815b909dc2eb1ce8403ee26d78f38a637 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:40.611 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:18:40.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:40.650 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:18:40.650 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:18:40.650 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:18:41.116 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:40 smithi057 systemd[1]: Stopping Ceph osd.3 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:41.369 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T03:18:41.115+0000 7fd9c8200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:41.369 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T03:18:41.115+0000 7fd9c8200640 -1 osd.3 114 *** Got signal Terminated *** 2024-09-12T03:18:41.370 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:41 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3[50750]: 2024-09-12T03:18:41.115+0000 7fd9c8200640 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:45.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:44 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:44.969+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:46.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:45.799+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:46.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:45 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:45.947+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:46.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:46.036+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:46.394 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:46.261+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:46.869 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:46 smithi057 podman[93208]: 2024-09-12 03:18:46.825389004 +0000 UTC m=+5.883098340 container died 899efbaae52c671efe0a57310677804938a5e80d40ccd9818aae4c461e3f9adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:18:47.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:46 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:46.753+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:47.278 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:46 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:46.927+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:47.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:47.045+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:47.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:47.260+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:47.572 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:47 smithi057 podman[93208]: 2024-09-12 03:18:47.279189862 +0000 UTC m=+6.336899195 container cleanup 899efbaae52c671efe0a57310677804938a5e80d40ccd9818aae4c461e3f9adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:18:47.572 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:47 smithi057 bash[93208]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:18:47.869 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:47 smithi057 podman[93226]: 2024-09-12 03:18:47.572491806 +0000 UTC m=+0.710532264 container remove 899efbaae52c671efe0a57310677804938a5e80d40ccd9818aae4c461e3f9adc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:18:48.142 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:47 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:47.885+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:48.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:47 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:47.776+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:48.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:48.058+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:48.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:48.221+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:48.567 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.048039434 +0000 UTC m=+0.027805590 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:48.567 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.21532839 +0000 UTC m=+0.195094536 container create 3537e873ab2e87df9cc83e84e54d8041c1b5caebf0ab85eb15a916aa7304c84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:18:48.842 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.568029254 +0000 UTC m=+0.547795422 container init 3537e873ab2e87df9cc83e84e54d8041c1b5caebf0ab85eb15a916aa7304c84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T03:18:48.842 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.575074881 +0000 UTC m=+0.554841027 container start 3537e873ab2e87df9cc83e84e54d8041c1b5caebf0ab85eb15a916aa7304c84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:18:48.842 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.667844708 +0000 UTC m=+0.647610858 container attach 3537e873ab2e87df9cc83e84e54d8041c1b5caebf0ab85eb15a916aa7304c84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:49.119 INFO:journalctl@ceph.osd.3.smithi057.stdout:Sep 12 03:18:48 smithi057 podman[93310]: 2024-09-12 03:18:48.843221911 +0000 UTC m=+0.822988070 container died 3537e873ab2e87df9cc83e84e54d8041c1b5caebf0ab85eb15a916aa7304c84c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T03:18:49.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:48 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:48.880+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:49.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:48 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:48.812+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:49.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:49.011+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:49.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:49.229+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:49.579 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:18:49.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:49.655 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:18:49.655 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:18:49.655 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:18:49.953 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:49 smithi181 systemd[1]: Stopping Ceph osd.4 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:49.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:49 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:49.819+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:50.272 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:50.001+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:50.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:49 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:49.897+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:50.453 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:49.999+0000 7f7217129640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:50.453 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:49.999+0000 7f7217129640 -1 osd.4 114 *** Got signal Terminated *** 2024-09-12T03:18:50.453 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:49.999+0000 7f7217129640 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:50.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:50.273+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:51.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:50 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:50.812+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:51.293 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:51.041+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:51.293 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:51.168+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:51.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:50 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:50.921+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:51.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:51.293+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:51.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:51 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:51.834+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:51.953 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:51 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:51.590+0000 7f721373f640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.505958+0000 front 2024-09-12T03:18:25.505986+0000 (oldest deadline 2024-09-12T03:18:50.806079+0000) 2024-09-12T03:18:52.330 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:52.120+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:52.330 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:52.063+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:52.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:51 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:51.969+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:52.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:51 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:51.935+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:52.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:52.330+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:52.954 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:52.841+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:52.954 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:52 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:52.600+0000 7f721373f640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.505958+0000 front 2024-09-12T03:18:25.505986+0000 (oldest deadline 2024-09-12T03:18:50.806079+0000) 2024-09-12T03:18:53.382 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:52.936+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:53.382 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:52 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:52.978+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:53.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:53.313+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:53.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:53.084+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:53.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:53.096+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:53.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:53 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:53.096+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:53.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:53.861+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:53.954 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:53 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:53.645+0000 7f721373f640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.505958+0000 front 2024-09-12T03:18:25.505986+0000 (oldest deadline 2024-09-12T03:18:50.806079+0000) 2024-09-12T03:18:54.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:53.952+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:54.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:53 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:53.964+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:54.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:54.270+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:54.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:54.270+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:54.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:54.087+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:54.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:54.108+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:54.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:54 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:54.108+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:54.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:54 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:54.393+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:54.703 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:54 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4[46698]: 2024-09-12T03:18:54.630+0000 7f721373f640 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.505958+0000 front 2024-09-12T03:18:25.505986+0000 (oldest deadline 2024-09-12T03:18:50.806079+0000) 2024-09-12T03:18:55.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:54 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:54.896+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:55.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:54.952+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:55.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:54.932+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:55.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:54 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:54.932+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:18:55.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:55 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:55.290+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:55.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:55 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:55.290+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:55.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:55 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:55.064+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:55.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:55 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:55.114+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:55.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:55 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:55.114+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:55.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:55 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:55.423+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:55.703 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:55 smithi181 podman[71763]: 2024-09-12 03:18:55.531630339 +0000 UTC m=+5.677968422 container died faff571a547651e87b0901a0ddd0e9c21b12db59bdc18298d1bf90cdff90bfe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:18:56.192 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:55 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:55.877+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:56.192 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:55 smithi181 podman[71763]: 2024-09-12 03:18:55.949651725 +0000 UTC m=+6.095989808 container cleanup faff571a547651e87b0901a0ddd0e9c21b12db59bdc18298d1bf90cdff90bfe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:18:56.192 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:55 smithi181 bash[71763]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:18:56.338 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:56 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:56.078+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:56.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:56 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:56.096+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:56.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:56 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:56.096+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:56.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:55.928+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:56.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:55.950+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:56.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:55 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:55.950+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:18:56.453 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:56 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:56.395+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:56.453 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:56 smithi181 podman[71778]: 2024-09-12 03:18:56.192721822 +0000 UTC m=+0.665292609 container remove faff571a547651e87b0901a0ddd0e9c21b12db59bdc18298d1bf90cdff90bfe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:18:56.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:56 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:56.338+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:56.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:56 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:56.338+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:56.885 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:56 smithi181 podman[71868]: 2024-09-12 03:18:56.513798989 +0000 UTC m=+0.015896707 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:18:56.885 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:56 smithi181 podman[71868]: 2024-09-12 03:18:56.643782197 +0000 UTC m=+0.145879915 container create 0a23d115502ab50eb4eb2ae8690d40e36511c951e57510763c78e20a57ce0859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:18:57.203 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:56 smithi181 podman[71868]: 2024-09-12 03:18:56.924925733 +0000 UTC m=+0.427023469 container init 0a23d115502ab50eb4eb2ae8690d40e36511c951e57510763c78e20a57ce0859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:18:57.204 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:56 smithi181 podman[71868]: 2024-09-12 03:18:56.928546105 +0000 UTC m=+0.430643825 container start 0a23d115502ab50eb4eb2ae8690d40e36511c951e57510763c78e20a57ce0859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:18:57.204 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:57 smithi181 podman[71868]: 2024-09-12 03:18:57.029968168 +0000 UTC m=+0.532065902 container attach 0a23d115502ab50eb4eb2ae8690d40e36511c951e57510763c78e20a57ce0859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T03:18:57.204 INFO:journalctl@ceph.osd.4.smithi181.stdout:Sep 12 03:18:57 smithi181 podman[71868]: 2024-09-12 03:18:57.126267972 +0000 UTC m=+0.628365693 container died 0a23d115502ab50eb4eb2ae8690d40e36511c951e57510763c78e20a57ce0859 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:18:57.204 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:56 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:56.885+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:57.204 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:56 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:56.885+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:18:57.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:56.920+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:57.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:56.938+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:57.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:56.938+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:18:57.375 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:57 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:57.053+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:57.375 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:57 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:57.089+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:57.375 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:57 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:57.089+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:57.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:57 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:57.375+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:57.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:57 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:57.375+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:57.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:57 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:57.430+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:57.769 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:18:57.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:18:57.808 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:18:57.808 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:18:57.808 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:18:58.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:57 smithi204 systemd[1]: Stopping Ceph osd.5 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:18:58.122 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:58.018+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:58.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:57 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:57.857+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:58.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:57 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:57.857+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:18:58.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:57.947+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:58.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:57.936+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:58.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:57 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:57.936+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:18:58.383 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:58.121+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:58.383 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:58.121+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:58.383 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:58.124+0000 7f1772406640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:18:58.385 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:58.124+0000 7f1772406640 -1 osd.5 114 *** Got signal Terminated *** 2024-09-12T03:18:58.385 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:58.124+0000 7f1772406640 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:18:58.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:58.383+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:58.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:58 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:58.383+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:58.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:58 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:58.415+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:59.205 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:58 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:58.889+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:18:59.205 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:58 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:58.889+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:18:59.348 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:18:59.045+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:18:59.348 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:59.114+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:18:59.349 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:18:59.115+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:18:59.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:58.942+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:18:59.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:58.978+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:18:59.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:18:58 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:18:58.978+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:18:59.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:59.348+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:18:59.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:59.348+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:18:59.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:18:59 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:18:59.348+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.809197+0000 front 2024-09-12T03:18:33.809280+0000 (oldest deadline 2024-09-12T03:18:58.509375+0000) 2024-09-12T03:18:59.937 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:59.457+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:18:59.938 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:18:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:18:59.457+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:00.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:59.938+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:00.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:18:59 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:18:59.938+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:00.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:18:59 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:18:59.932+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:00.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:00.005+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:00.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:00.005+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:00.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:00.030+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:00.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:00.120+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:00.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:00.120+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:00.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:00.120+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:00.893 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:00.396+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:19:00.893 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:00.396+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:19:00.893 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:00 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:00.396+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.809197+0000 front 2024-09-12T03:18:33.809280+0000 (oldest deadline 2024-09-12T03:18:58.509375+0000) 2024-09-12T03:19:00.900 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:00 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:00.471+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:00.900 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:00 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:00.471+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:01.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:00 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:00.900+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:01.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:00 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:00.900+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:01.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:00 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:00.900+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:01.357 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:01.038+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:01.357 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:01.038+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:01.357 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:01.141+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:01.357 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:01.141+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:01.357 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:01.141+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:01.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:00.975+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:01.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:00 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:00.975+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:01.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.000+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:01.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.000+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:01.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.000+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:01.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:01.356+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:19:01.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:01.356+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:19:01.643 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:01 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:01.356+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.809197+0000 front 2024-09-12T03:18:33.809280+0000 (oldest deadline 2024-09-12T03:18:58.509375+0000) 2024-09-12T03:19:01.949 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:01 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:01.514+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:01.950 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:01 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:01.514+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:02.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:01 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:01.949+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:02.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:01 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:01.949+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:02.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:01 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:01.949+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:02.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:02.052+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:02.364 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:02.052+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:02.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:02.190+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:02.364 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:02.190+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:02.365 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:02.190+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:02.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:02 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:02.023+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:02.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:02 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:02.023+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:02.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.984+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:02.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.984+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:02.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:01.984+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:02.640 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:02.363+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:15.607499+0000 front 2024-09-12T03:18:15.607588+0000 (oldest deadline 2024-09-12T03:18:46.207828+0000) 2024-09-12T03:19:02.640 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:02.363+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:29.109132+0000 front 2024-09-12T03:18:29.109070+0000 (oldest deadline 2024-09-12T03:18:53.808985+0000) 2024-09-12T03:19:02.640 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:02 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5[46861]: 2024-09-12T03:19:02.363+0000 7f176ea1c640 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.809197+0000 front 2024-09-12T03:18:33.809280+0000 (oldest deadline 2024-09-12T03:18:58.509375+0000) 2024-09-12T03:19:02.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:02 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:02.521+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:02.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:02 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:02.521+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:03.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:02 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:02.943+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:03.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:02 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:02.943+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:03.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:02 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:02.943+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:03.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:03 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:03.056+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:03.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:03 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:03.056+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:03.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:03 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:03.021+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:03.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:03 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:03.021+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:03.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:03 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:03.021+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:03.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:03 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:03.070+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:03.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:03 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:03.070+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:03.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:03 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:03.206+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:03.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:03 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:03.206+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:03.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:03 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:03.206+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:03.893 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:03 smithi204 podman[72064]: 2024-09-12 03:19:03.841441898 +0000 UTC m=+5.838556927 container died 5977f3161707f69cac655ff098781b89bb20081fdd877fbdc4e09bf09cf5b11e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:19:03.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:03 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:03.478+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:03.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:03 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:03.478+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:04.271 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:04.045+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:04.271 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:04.045+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:04.271 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:04.017+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:04.272 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:04.017+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:04.272 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:04 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:04.017+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:04.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:04 smithi204 podman[72064]: 2024-09-12 03:19:04.241776515 +0000 UTC m=+6.238891543 container cleanup 5977f3161707f69cac655ff098781b89bb20081fdd877fbdc4e09bf09cf5b11e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:19:04.393 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:04 smithi204 bash[72064]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:19:04.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:04.052+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:04.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:04.052+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:04.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:04.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:04.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:04.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:04.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:04 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:04.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:04.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:03 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:03.992+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:04.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:03 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:03.992+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:04.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:03 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:03.992+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:04.854 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:04 smithi204 podman[72091]: 2024-09-12 03:19:04.51878306 +0000 UTC m=+0.680281925 container remove 5977f3161707f69cac655ff098781b89bb20081fdd877fbdc4e09bf09cf5b11e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:19:04.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:04 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:04.506+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:04.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:04 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:04.506+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:05.138 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:05 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:05.039+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:05.138 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:05 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:05.039+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:05.139 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:04 smithi204 podman[72181]: 2024-09-12 03:19:04.757419899 +0000 UTC m=+0.015841598 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:05.139 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:04 smithi204 podman[72181]: 2024-09-12 03:19:04.886324055 +0000 UTC m=+0.144745747 container create 620f6aca4a3341460a6bd1e771b2103ba573174fd4e6d68d006fdd8f32bb8196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:19:05.139 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:05 smithi204 podman[72181]: 2024-09-12 03:19:05.139210446 +0000 UTC m=+0.397632138 container init 620f6aca4a3341460a6bd1e771b2103ba573174fd4e6d68d006fdd8f32bb8196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:19:05.295 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:05.072+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:05.296 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:05.072+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:05.296 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.039+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:05.296 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.039+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:05.296 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.039+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:05.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:05 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:05.248+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:05.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:05 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:05.248+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:05.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:05 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:05.248+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:05.394 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:05 smithi204 podman[72181]: 2024-09-12 03:19:05.142631714 +0000 UTC m=+0.401053407 container start 620f6aca4a3341460a6bd1e771b2103ba573174fd4e6d68d006fdd8f32bb8196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:05.394 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:05 smithi204 podman[72181]: 2024-09-12 03:19:05.230963253 +0000 UTC m=+0.489384947 container attach 620f6aca4a3341460a6bd1e771b2103ba573174fd4e6d68d006fdd8f32bb8196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:19:05.394 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 12 03:19:05 smithi204 podman[72181]: 2024-09-12 03:19:05.338382276 +0000 UTC m=+0.596803971 container died 620f6aca4a3341460a6bd1e771b2103ba573174fd4e6d68d006fdd8f32bb8196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:05.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:04 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:04.970+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:05.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:04 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:04.970+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:05.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:04 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:04.970+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:05.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:05 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:05.511+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:05.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:05 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:05.511+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:06.012 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:19:06.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:06.053 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:19:06.054 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:19:06.054 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:19:06.256 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:06 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:06.026+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:06.256 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:06 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:06.026+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:06.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:06.090+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:06.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:06.090+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:06.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 systemd[1]: Stopping Ceph osd.6 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:06.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.995+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:06.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.995+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:06.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:05 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:05.995+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:06.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.016+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:06.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.016+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:06.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.016+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:06.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:06 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:06.256+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:06.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:06 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:06.256+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:06.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:06 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:06.256+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:06.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:06.532+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:06.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:06.532+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:07.029 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:06.643+0000 7fedbf000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:07.029 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:06.643+0000 7fedbf000640 -1 osd.6 114 *** Got signal Terminated *** 2024-09-12T03:19:07.029 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:06.643+0000 7fedbf000640 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:07.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:07.102+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:07.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:07.102+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:07.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:07.028+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:07.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:07.028+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:07.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:07 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:07.028+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:07.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:07 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:07.061+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:07.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:07 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:07.061+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:07.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:07 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:07.230+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:07.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:07 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:07.230+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:07.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:07 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:07.230+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:07.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.994+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:07.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.994+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:07.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:06 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:06.994+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:07.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:07 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:07.564+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:07.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:07 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:07.564+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:08.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:08.071+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:08.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:08.071+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:08.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:08.056+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:08.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:08.056+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:08.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:08 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:08.056+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:08.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:08.101+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:08.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:08.101+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:08.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:08.237+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:08.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:08.237+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:08.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:08 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:08.237+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:08.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:08.023+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:08.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:08.023+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:08.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:08.023+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:08.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:08.023+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:08.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:08.525+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:08.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:08 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:08.525+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:09.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:09.044+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:09.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:09.044+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:09.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:09.007+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:09.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:09.007+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:09.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:09 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:09.007+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:09.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:09.059+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:09.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:09.059+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:09.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:09.059+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:09.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:09.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:09.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:09.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:09.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:09 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:09.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:09.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:09.001+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:09.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:09.001+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:09.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:09.001+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:09.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:09.001+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:09.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:09.557+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:09.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:09.557+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:09.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:09 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:09.557+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:10.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:10.046+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:10.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:10.046+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:10.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:10.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:10.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:10.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:10.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:10.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:10.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:10 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:10.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:10.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:10.090+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:10.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:10.090+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:10.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:10.090+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:10.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:10.242+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:10.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:10.242+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:10.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:10 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:10.242+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:10.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:10.048+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:10.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:10.048+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:10.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:10.048+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:10.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:10.048+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:10.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:10.525+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:10.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:10.525+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:10.954 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:10 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:10.525+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:11.085+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:26.226358+0000 front 2024-09-12T03:18:26.226482+0000 (oldest deadline 2024-09-12T03:18:50.926545+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:11.085+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:35.027903+0000 front 2024-09-12T03:18:35.027994+0000 (oldest deadline 2024-09-12T03:19:00.928064+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6[57690]: 2024-09-12T03:19:11.085+0000 7fedbaa00640 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.930277+0000 front 2024-09-12T03:18:44.930318+0000 (oldest deadline 2024-09-12T03:19:10.730919+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:11.099+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:11.099+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:11.099+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:11.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:11.099+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:11.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:11.140+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:11.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:11.140+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:11.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:11.140+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:11.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:11.234+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:11.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:11.234+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:11.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:11 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:11.234+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:11.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:11.032+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:11.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:11.032+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:11.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:11.032+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:11.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:11.032+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:11.939 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:11.495+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:11.940 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:11.495+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:11.940 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:11 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:11.495+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:12.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:12.033+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:12.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:12.033+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:12.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:12.033+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:12.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:12.033+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:12.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:12 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:12.078+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:12.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:12 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:12.078+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:12.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:12 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:12.078+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:12.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:12 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:12.078+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:12.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:12.100+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:12.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:12.100+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:12.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:12.100+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:12.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:12.268+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:12.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:12.268+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:12.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:12 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:12.268+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:12.759 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:12 smithi057 podman[93422]: 2024-09-12 03:19:12.397336923 +0000 UTC m=+6.012663647 container died a141222e7ee44f46cf74efd437e99442fd523720918c8a5de1470e021b1edff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:19:12.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:12.454+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:12.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:12.455+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:12.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:12 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:12.455+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:13.086 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:12 smithi057 podman[93422]: 2024-09-12 03:19:12.865988085 +0000 UTC m=+6.481314809 container cleanup a141222e7ee44f46cf74efd437e99442fd523720918c8a5de1470e021b1edff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:19:13.086 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:12 smithi057 bash[93422]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:19:13.369 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:13 smithi057 podman[93439]: 2024-09-12 03:19:13.086681786 +0000 UTC m=+0.665838675 container remove a141222e7ee44f46cf74efd437e99442fd523720918c8a5de1470e021b1edff8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:19:13.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:13.101+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:13.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:13.101+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:13.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:13.101+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:13.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:13 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:13.101+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:13.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:13.067+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:13.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:13.067+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:13.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:13.067+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:13.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:13.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:13.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:13.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:13.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:13 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:13.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:13.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:13.058+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:13.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:13.058+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:13.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:13.058+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:13.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:13.058+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:13.710 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:13.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:13.743 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:13.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:13.743 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:13 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:13.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:13.970 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:13 smithi057 podman[93525]: 2024-09-12 03:19:13.54823068 +0000 UTC m=+0.027689764 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:13.971 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:13 smithi057 podman[93525]: 2024-09-12 03:19:13.676729109 +0000 UTC m=+0.156188236 container create bc49a833495ae1de03a1a3b7af7540f7121b3213c274120634958a521909acd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:13.971 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:13 smithi057 podman[93525]: 2024-09-12 03:19:13.970883579 +0000 UTC m=+0.450342664 container init bc49a833495ae1de03a1a3b7af7540f7121b3213c274120634958a521909acd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:19:14.245 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:13 smithi057 podman[93525]: 2024-09-12 03:19:13.977836463 +0000 UTC m=+0.457295558 container start bc49a833495ae1de03a1a3b7af7540f7121b3213c274120634958a521909acd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:19:14.245 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:14 smithi057 podman[93525]: 2024-09-12 03:19:14.212438735 +0000 UTC m=+0.691897828 container attach bc49a833495ae1de03a1a3b7af7540f7121b3213c274120634958a521909acd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:19:14.245 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:14 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:14.094+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:14.245 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:14 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:14.094+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:14.246 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:14 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:14.094+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:14.246 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:14 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:14.094+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:14.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:14.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:14.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:14.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:14.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:14.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:14.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:14.092+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:14.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:14.092+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:14.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:14 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:14.092+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:14.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:14.106+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:14.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:14.106+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:14.444 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:14.106+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:14.445 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:14.106+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:14.619 INFO:journalctl@ceph.osd.6.smithi057.stdout:Sep 12 03:19:14 smithi057 podman[93547]: 2024-09-12 03:19:14.293232018 +0000 UTC m=+0.031674607 container died bc49a833495ae1de03a1a3b7af7540f7121b3213c274120634958a521909acd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:19:14.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:14.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:14.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:14.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:14.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:14 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:14.444+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:14.975 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:19:15.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:15.051 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:19:15.051 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:19:15.051 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:19:15.361 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 systemd[1]: Stopping Ceph osd.7 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:15.361 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.361+0000 7f7fb6ad2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:15.361 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.361+0000 7f7fb6ad2640 -1 osd.7 114 *** Got signal Terminated *** 2024-09-12T03:19:15.361 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.361+0000 7f7fb6ad2640 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:15.362 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:15.149+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:15.362 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:15.149+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:15.362 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:15.149+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:15.362 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:15.149+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:15.373 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:15.100+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:15.373 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:15.100+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:15.373 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:15.100+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:15.373 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:15 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:15.100+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:15.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:15.099+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:15.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:15.099+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:15.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:15.099+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:15.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:15.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:15.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:15.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:15.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:15 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:15.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:15.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:15.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:15.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:15 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:15.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:16.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:16.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:16.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:16.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:16.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:16.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:16.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:16 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:16.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:16.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:16.296+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:16.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:16.296+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:16.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:16.296+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:16.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:16.063+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:16.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:16.063+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:16.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:16.063+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:16.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:16 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:16.063+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:16.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:16.122+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:16.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:16.122+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:16.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:16.122+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:16.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:16.122+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:16.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:16.484+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:16.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:16.484+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:16.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:16.484+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:17.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:17.060+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:17.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:17.060+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:17.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:17.060+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:17.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:17 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:17.060+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:17.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:17.107+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:17.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:17.107+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:17.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:17.107+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:17.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:17.107+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:17.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:17.317+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:17.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:17.317+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:17.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:17.317+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:17.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:17 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:17.317+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:17.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:17.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:17.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:17.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:17.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:17.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:17.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:17.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:17.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:17.468+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:17.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:17.468+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:17.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:17 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:17.468+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:18.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:18 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:18.067+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:18.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:18 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:18.067+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:18.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:18 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:18.067+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:18.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:18 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:18.067+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:18.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:18.085+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:18.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:18.085+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:18.085+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:18.085+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:18.272+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:18.272+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:18.272+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:18.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:18 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:18.272+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:18.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:18.057+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:18.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:18.057+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:18.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:18.057+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:18.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:18.057+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:18.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:18.057+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:18.955 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:18.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:18.955 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:18.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:18.955 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:18 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:18.462+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:19.089 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:19.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:19.089 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:19.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:19.089 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:19.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:19.089 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:19.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:19.089 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:19.085+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:19.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:19 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:19.027+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:19.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:19 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:19.027+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:19.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:19 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:19.027+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:19.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:19 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:19.027+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:19.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:19 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:19.027+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:19.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:19.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:19.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:19.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:19.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:19.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:19.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:19.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:19.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:19.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:19.395 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:19.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:19.395 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:19.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:19.395 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:19 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:19.249+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:19.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:19.465+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:27.583317+0000 front 2024-09-12T03:18:27.583361+0000 (oldest deadline 2024-09-12T03:18:53.483398+0000) 2024-09-12T03:19:19.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:19.465+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.584601+0000 front 2024-09-12T03:18:34.584551+0000 (oldest deadline 2024-09-12T03:18:58.684544+0000) 2024-09-12T03:19:19.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:19.465+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.986880+0000 front 2024-09-12T03:18:44.986809+0000 (oldest deadline 2024-09-12T03:19:09.086589+0000) 2024-09-12T03:19:19.954 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:19 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7[53660]: 2024-09-12T03:19:19.465+0000 7f7fb30e8640 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:54.387459+0000 front 2024-09-12T03:18:54.387555+0000 (oldest deadline 2024-09-12T03:19:19.087552+0000) 2024-09-12T03:19:20.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:20 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:20.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:20.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:20 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:20.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:20.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:20 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:20.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:20.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:20 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:20.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:20.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:20 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:20.052+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:20.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:20.091+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:20.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:20.091+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:20.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:20.091+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:20.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:20.091+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:20.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:20.290+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:20.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:20.290+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:20.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:20.290+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:20.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:20 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:20.290+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:20.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:20.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:20.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:20.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:20.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:20.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:20.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:20.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:20.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:20 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:20.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:21.278 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:20 smithi181 podman[71981]: 2024-09-12 03:19:20.954273987 +0000 UTC m=+5.706547923 container died d8ec301c83b027fff8cafc91d99a2aebaff15e3bf7c07e7e4813b89896af9d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:19:21.279 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:21 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:21.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:21.279 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:21 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:21.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:21.279 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:21 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:21.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:21.279 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:21 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:21.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:21.279 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:21 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:21.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:21.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:21 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:21.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:21.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:21 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:21.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:21.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:21 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:21.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:21.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:21 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:21.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:21.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:21 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:21.074+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:21.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:21.124+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:21.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:21.124+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:21.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:21.124+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:21.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:21.124+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:21.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:21.273+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:21.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:21.273+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:21.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:21.273+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:21.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:21 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:21.273+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:21.587 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:21 smithi181 podman[71981]: 2024-09-12 03:19:21.386226345 +0000 UTC m=+6.138500285 container cleanup d8ec301c83b027fff8cafc91d99a2aebaff15e3bf7c07e7e4813b89896af9d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:19:21.587 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:21 smithi181 bash[71981]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:19:21.953 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:21 smithi181 podman[71997]: 2024-09-12 03:19:21.587534518 +0000 UTC m=+0.635260808 container remove d8ec301c83b027fff8cafc91d99a2aebaff15e3bf7c07e7e4813b89896af9d28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:19:22.298 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:21 smithi181 podman[72086]: 2024-09-12 03:19:21.892338209 +0000 UTC m=+0.015499078 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:22 smithi181 podman[72086]: 2024-09-12 03:19:22.038588017 +0000 UTC m=+0.161748932 container create b5551c7a97897e61089d08c8160508ba94f89a374b626a6e1ae598aec47ff60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:22.108+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:22.108+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:22.108+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:22.108+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:22.299 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:22.108+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:22.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:22 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:22.091+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:22.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:22 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:22.091+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:22.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:22 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:22.091+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:22.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:22 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:22.091+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:22.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:22 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:22.091+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:22.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:22.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:22.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:22.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:22.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:22.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:22.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:22.079+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:22.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:22.308+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:22.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:22.308+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:22.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:22.308+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:22.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:22 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:22.308+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:22 smithi181 podman[72086]: 2024-09-12 03:19:22.299318384 +0000 UTC m=+0.422479254 container init b5551c7a97897e61089d08c8160508ba94f89a374b626a6e1ae598aec47ff60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:19:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:22 smithi181 podman[72086]: 2024-09-12 03:19:22.302672414 +0000 UTC m=+0.425833288 container start b5551c7a97897e61089d08c8160508ba94f89a374b626a6e1ae598aec47ff60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T03:19:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:22 smithi181 podman[72086]: 2024-09-12 03:19:22.415994958 +0000 UTC m=+0.539155843 container attach b5551c7a97897e61089d08c8160508ba94f89a374b626a6e1ae598aec47ff60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:19:22.703 INFO:journalctl@ceph.osd.7.smithi181.stdout:Sep 12 03:19:22 smithi181 podman[72086]: 2024-09-12 03:19:22.488884302 +0000 UTC m=+0.612045172 container died b5551c7a97897e61089d08c8160508ba94f89a374b626a6e1ae598aec47ff60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:19:23.119 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:19:23.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:23.158 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:19:23.159 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:19:23.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:19:23.366 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:23 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:23.155+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:23.366 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:23 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:23.155+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:23.366 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:23 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:23.155+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:23.366 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:23 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:23.155+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:23.366 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:23 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:23.155+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:23.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:23 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:23.093+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:23.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:23 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:23.093+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:23.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:23 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:23.093+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:23.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:23 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:23.093+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:23.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:23 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:23.093+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:23.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:23.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:23.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:23.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:23.293+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.072+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.072+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.072+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.072+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:23.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 systemd[1]: Stopping Ceph osd.8 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:23.893 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.471+0000 7fc8d0d8d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:23.893 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.471+0000 7fc8d0d8d640 -1 osd.8 114 *** Got signal Terminated *** 2024-09-12T03:19:23.893 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:23 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:23.471+0000 7fc8d0d8d640 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:24.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:24 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:24.110+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:24.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:24 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:24.110+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:24.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:24 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:24.110+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:24.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:24 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:24.110+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:24.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:24 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:24.110+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:24.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:24.260+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:24.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:24.260+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:24.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:24.260+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:24.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:24.260+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:24.260+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:24.102+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:24.102+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:24.102+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:24.102+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:24.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:24 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:24.102+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:59.394104+0000 front 2024-09-12T03:18:59.394128+0000 (oldest deadline 2024-09-12T03:19:24.094183+0000) 2024-09-12T03:19:24.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:24.203+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:24.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:24.203+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:24.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:24.203+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:24.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:24.203+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:24.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:24.203+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:25.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:25 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:25.068+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:25.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:25 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:25.068+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:25.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:25 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:25.068+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:25.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:25 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:25.068+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:25.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:25 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:25.068+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:25.393 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:25.121+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:25.121+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:25.121+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:25.121+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:25.121+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:59.394104+0000 front 2024-09-12T03:18:59.394128+0000 (oldest deadline 2024-09-12T03:19:24.094183+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:25.277+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:25.277+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:25.277+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:25.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:25.277+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:25.395 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:25 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:25.277+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:25.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:25 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:25.240+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:26.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:26 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:26.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:26.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:26 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:26.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:26.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:26 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:26.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:26.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:26 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:26.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:26.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:26 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:26.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:26.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:26.148+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:26.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:26.148+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:26.148+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:26.148+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:26.148+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:59.394104+0000 front 2024-09-12T03:18:59.394128+0000 (oldest deadline 2024-09-12T03:19:24.094183+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:26.269+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:26.269+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:26.269+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:26.269+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:26.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:26 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:26.269+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:26.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:26 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:26.236+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:27.619 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:27 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:27.128+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:27.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:27 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:27.128+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:27.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:27 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:27.128+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:27.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:27 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:27.128+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:27.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:27 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:27.128+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:27.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:27.179+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:27.179+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:27.179+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:27.179+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:27.179+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:59.394104+0000 front 2024-09-12T03:18:59.394128+0000 (oldest deadline 2024-09-12T03:19:24.094183+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:27.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:27.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:27.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:27.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:27.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:27 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:27.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:27.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:27.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:27.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:27.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:27.704 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:27.704 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:27 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:27.266+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:28.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:28.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:28.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:28.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:28.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:28.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:28.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:28.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:28.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:28 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:28.103+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:28.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:28.222+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.787984+0000 front 2024-09-12T03:18:28.787924+0000 (oldest deadline 2024-09-12T03:18:51.087966+0000) 2024-09-12T03:19:28.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:28.222+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.189743+0000 front 2024-09-12T03:18:36.189846+0000 (oldest deadline 2024-09-12T03:19:00.289972+0000) 2024-09-12T03:19:28.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:28.222+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.792056+0000 front 2024-09-12T03:18:44.791914+0000 (oldest deadline 2024-09-12T03:19:08.892016+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:28.222+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:49.993046+0000 front 2024-09-12T03:18:49.993057+0000 (oldest deadline 2024-09-12T03:19:15.893071+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8[53520]: 2024-09-12T03:19:28.222+0000 7fc8c939b640 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:59.394104+0000 front 2024-09-12T03:18:59.394128+0000 (oldest deadline 2024-09-12T03:19:24.094183+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:28.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:28.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:28.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:28.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:28.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:28 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:28.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:28.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:28.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:28.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:28.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:28.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:28.704 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:28 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:28.264+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:29.300 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:29 smithi204 podman[72286]: 2024-09-12 03:19:29.028713319 +0000 UTC m=+5.668079669 container died bae891b3ea966e95a4039a2012011023e4e0c4de10cdea4b206f385c11410ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:29.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:29 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:29.140+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:29.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:29 smithi204 podman[72286]: 2024-09-12 03:19:29.421763986 +0000 UTC m=+6.061130323 container cleanup bae891b3ea966e95a4039a2012011023e4e0c4de10cdea4b206f385c11410ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:19:29.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:29 smithi204 bash[72286]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:19:29.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:29 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:29.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:29.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:29 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:29.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:29.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:29 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:29.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:29.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:29 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:29.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:29.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:29 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:29.300+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:29.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:29.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:29.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:29.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:29.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:29.704 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:29.273+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:30.035 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:29 smithi204 podman[72301]: 2024-09-12 03:19:29.689977677 +0000 UTC m=+0.656545033 container remove bae891b3ea966e95a4039a2012011023e4e0c4de10cdea4b206f385c11410ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:19:30.287 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:29.938859057 +0000 UTC m=+0.015712319 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:30.04916036 +0000 UTC m=+0.126013662 container create 627d16f8ec48794bcfcb632b861ce918d34b219c0341ff0a71a69bd84c1eea13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:30.28497089 +0000 UTC m=+0.361824151 container init 627d16f8ec48794bcfcb632b861ce918d34b219c0341ff0a71a69bd84c1eea13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:30 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:30.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:30 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:30.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:30 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:30.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:30 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:30.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:30.288 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:30 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:30.255+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:30.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:30 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:30.097+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:30.604 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:30.604 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:30.605 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:30.605 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:30.605 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:30.605 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:30 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:30.225+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:30.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:30.288244407 +0000 UTC m=+0.365097663 container start 627d16f8ec48794bcfcb632b861ce918d34b219c0341ff0a71a69bd84c1eea13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:30.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:30.401999366 +0000 UTC m=+0.478852626 container attach 627d16f8ec48794bcfcb632b861ce918d34b219c0341ff0a71a69bd84c1eea13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T03:19:30.643 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 12 03:19:30 smithi204 podman[72393]: 2024-09-12 03:19:30.478988667 +0000 UTC m=+0.555841924 container died 627d16f8ec48794bcfcb632b861ce918d34b219c0341ff0a71a69bd84c1eea13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:19:31.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:19:31.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:31.201 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:19:31.201 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:19:31.202 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:19:31.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:31 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:31.265+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:31.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:31 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:31.265+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:31.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:31 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:31.265+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:31.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:31 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:31.265+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:31.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:31 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:31.265+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:31.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:31 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:31.201+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:31.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:31.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:31.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:31.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:31.620 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:31.621 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.119+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:31.621 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 systemd[1]: Stopping Ceph osd.9 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:32.076 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.656+0000 7f3fea000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:32.077 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.656+0000 7f3fea000640 -1 osd.9 114 *** Got signal Terminated *** 2024-09-12T03:19:32.077 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:31 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:31.656+0000 7f3fea000640 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:32.077 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:32.077 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:32.369 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:32.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:32.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:32.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:32 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:32.076+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:32.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:32 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:32.191+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:32.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:32.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:32.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:32.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:32.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:32.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:32.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:32.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:32.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:32 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:32.278+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:33.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:33 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:33.104+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:33.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:33.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:33.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:33.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:33.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:33.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:33.295+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:33.703 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:33 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:33.229+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:34.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:34.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:34.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:34.371 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:34.371 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:34.371 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:34 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:34.079+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:34.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:34 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:34.189+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:34.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:34.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:34.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:34.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:34.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:34.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:34 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:34.331+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:35.370 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:35 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:35.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:35.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:35 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:35.150+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:35.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:35.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:35.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:35.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:35.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:35.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:35 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:35.369+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:19.399682+0000 front 2024-09-12T03:18:19.399804+0000 (oldest deadline 2024-09-12T03:18:44.099770+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:24.100632+0000 front 2024-09-12T03:18:24.100525+0000 (oldest deadline 2024-09-12T03:18:54.701180+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:34.701684+0000 front 2024-09-12T03:18:34.701858+0000 (oldest deadline 2024-09-12T03:19:00.502392+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:44.504088+0000 front 2024-09-12T03:18:44.504143+0000 (oldest deadline 2024-09-12T03:19:09.204123+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:52.606193+0000 front 2024-09-12T03:18:52.606166+0000 (oldest deadline 2024-09-12T03:19:18.506262+0000) 2024-09-12T03:19:36.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:36 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9[64266]: 2024-09-12T03:19:36.064+0000 7f3fe5a00640 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:18:58.507228+0000 front 2024-09-12T03:18:58.507062+0000 (oldest deadline 2024-09-12T03:19:29.107917+0000) 2024-09-12T03:19:36.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:36.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:36.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:36.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:36.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:36.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:36.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:36 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:36.130+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:36.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:36.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:36.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:36.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:36.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:36.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:36 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:36.328+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:37.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:37 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:37.147+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:37.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:37.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:37.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:37.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:37.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:37.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:37 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:37.340+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:37.836 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:37 smithi057 podman[93626]: 2024-09-12 03:19:37.43447323 +0000 UTC m=+5.941246514 container died b5f3c3207ccbe2acce6649c5481d010b2e9e2260151692661bfb0c205ce96535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:38.119 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:37 smithi057 podman[93626]: 2024-09-12 03:19:37.950697682 +0000 UTC m=+6.457470975 container cleanup b5f3c3207ccbe2acce6649c5481d010b2e9e2260151692661bfb0c205ce96535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:19:38.120 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:37 smithi057 bash[93626]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:38.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:38 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:38.135+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:38.619 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:38 smithi057 podman[93655]: 2024-09-12 03:19:38.260577543 +0000 UTC m=+0.805678481 container remove b5f3c3207ccbe2acce6649c5481d010b2e9e2260151692661bfb0c205ce96535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:19:38.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:38.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:38.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:38.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:38.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:38.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:38.333+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:39.013 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:38 smithi057 podman[93741]: 2024-09-12 03:19:38.627692208 +0000 UTC m=+0.027875586 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:39.013 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:38 smithi057 podman[93741]: 2024-09-12 03:19:38.794659781 +0000 UTC m=+0.194843149 container create 5e92e92b85d1752c1c821214c8899c01705f84e7ebe43b07c0c9ef046bfdab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:39.013 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:39 smithi057 podman[93741]: 2024-09-12 03:19:39.013615464 +0000 UTC m=+0.413798828 container init 5e92e92b85d1752c1c821214c8899c01705f84e7ebe43b07c0c9ef046bfdab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:19:39.289 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:39 smithi057 podman[93741]: 2024-09-12 03:19:39.020347691 +0000 UTC m=+0.420531057 container start 5e92e92b85d1752c1c821214c8899c01705f84e7ebe43b07c0c9ef046bfdab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:19:39.290 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:39 smithi057 podman[93741]: 2024-09-12 03:19:39.10495715 +0000 UTC m=+0.505140523 container attach 5e92e92b85d1752c1c821214c8899c01705f84e7ebe43b07c0c9ef046bfdab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:39.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:39.645 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:39.645 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:39.645 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:39.646 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:39.646 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:39.646 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:39 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:39.119+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:39 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:39.350+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:39.647 INFO:journalctl@ceph.osd.9.smithi057.stdout:Sep 12 03:19:39 smithi057 podman[93741]: 2024-09-12 03:19:39.290336958 +0000 UTC m=+0.690520321 container died 5e92e92b85d1752c1c821214c8899c01705f84e7ebe43b07c0c9ef046bfdab9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:40.094 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:19:40.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:40.171 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:19:40.171 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:19:40.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:19:40.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:40.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:40.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:40.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:40.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:40.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:40.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.129+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:40.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 systemd[1]: Stopping Ceph osd.10 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:40.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:40.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:40.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:40.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:40.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:40.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:40 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:40.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:40.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.480+0000 7f529790b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:40.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.480+0000 7f529790b640 -1 osd.10 114 *** Got signal Terminated *** 2024-09-12T03:19:40.953 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:40 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:40.480+0000 7f529790b640 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:41.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:41 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:41.143+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:41.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:41.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:41.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:41.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:41.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:41.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:41 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:41.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:42.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:42 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:42.101+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:42.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:42.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:42.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:42.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:42.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:42.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:42.356+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:43.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:43.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:43 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:43.080+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:43.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:43.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:43.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:43.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:43.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:43.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:43.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:43 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:43.373+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:44.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:44.454 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:44 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:44.077+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:44.893 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:44.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:44 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:44.408+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:16.669132+0000 front 2024-09-12T03:18:16.669278+0000 (oldest deadline 2024-09-12T03:18:45.469595+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:25.469947+0000 front 2024-09-12T03:18:25.469965+0000 (oldest deadline 2024-09-12T03:18:56.070744+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:36.571594+0000 front 2024-09-12T03:18:36.571736+0000 (oldest deadline 2024-09-12T03:19:00.671636+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6814 osd.3 since back 2024-09-12T03:18:42.773837+0000 front 2024-09-12T03:18:42.773729+0000 (oldest deadline 2024-09-12T03:19:07.473753+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:53.374815+0000 front 2024-09-12T03:18:53.374779+0000 (oldest deadline 2024-09-12T03:19:17.474934+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:02.076549+0000 front 2024-09-12T03:19:02.076461+0000 (oldest deadline 2024-09-12T03:19:24.876713+0000) 2024-09-12T03:19:45.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10[60212]: 2024-09-12T03:19:45.060+0000 7f528ff19640 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:09.378421+0000 front 2024-09-12T03:19:09.378465+0000 (oldest deadline 2024-09-12T03:19:33.378651+0000) 2024-09-12T03:19:45.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:45.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:45.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:45.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:45.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:45.895 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:45.895 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:45 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:45.393+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:46.189 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:45 smithi181 podman[72201]: 2024-09-12 03:19:45.921385857 +0000 UTC m=+5.558118766 container died cb87ab63636dcf26876d8b338d9de0047fb21518a9659521e56a7e4a988781ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:19:46.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:46 smithi181 podman[72201]: 2024-09-12 03:19:46.271480581 +0000 UTC m=+5.908213488 container cleanup cb87ab63636dcf26876d8b338d9de0047fb21518a9659521e56a7e4a988781ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:19:46.453 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:46 smithi181 bash[72201]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:19:46.873 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:46 smithi181 podman[72216]: 2024-09-12 03:19:46.556280415 +0000 UTC m=+0.629736945 container remove cb87ab63636dcf26876d8b338d9de0047fb21518a9659521e56a7e4a988781ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:19:46.893 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:46.893 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:46.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:46.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:46.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:46.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:46.894 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:46.419+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:47.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:46 smithi181 podman[72304]: 2024-09-12 03:19:46.776332218 +0000 UTC m=+0.015784224 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:47.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:46 smithi181 podman[72304]: 2024-09-12 03:19:46.881941095 +0000 UTC m=+0.121393095 container create 33aff2350a9909426185461d9cbcb251778faa9327fe65a7f5660349fb7718b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:19:47.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:47 smithi181 podman[72304]: 2024-09-12 03:19:47.117669025 +0000 UTC m=+0.357121017 container init 33aff2350a9909426185461d9cbcb251778faa9327fe65a7f5660349fb7718b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:19:47.203 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:47 smithi181 podman[72304]: 2024-09-12 03:19:47.120990477 +0000 UTC m=+0.360442473 container start 33aff2350a9909426185461d9cbcb251778faa9327fe65a7f5660349fb7718b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:19:47.602 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:47 smithi181 podman[72304]: 2024-09-12 03:19:47.242539806 +0000 UTC m=+0.481991801 container attach 33aff2350a9909426185461d9cbcb251778faa9327fe65a7f5660349fb7718b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:19:47.602 INFO:journalctl@ceph.osd.10.smithi181.stdout:Sep 12 03:19:47 smithi181 podman[72304]: 2024-09-12 03:19:47.310507974 +0000 UTC m=+0.549959985 container died 33aff2350a9909426185461d9cbcb251778faa9327fe65a7f5660349fb7718b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:19:47.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:47.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:47.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:47.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:47.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:47.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:47.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:47.371+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:47.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:19:47.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:47.929 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:19:47.929 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:19:47.929 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:19:48.217 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:47 smithi204 systemd[1]: Stopping Ceph osd.11 for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:48.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.216+0000 7f9467c57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:48.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.216+0000 7f9467c57640 -1 osd.11 114 *** Got signal Terminated *** 2024-09-12T03:19:48.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.216+0000 7f9467c57640 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:19:48.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:48.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:48 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:48.359+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:49.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:49.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:49.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:49.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:49.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:49.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:49.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:49 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:49.313+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:50.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:50.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:50.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:50.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:50.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:50.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:50.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:50 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:50.343+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:51.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:51.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:51.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:51.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:51.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:51.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:51.645 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:51.646 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:51 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:51.364+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-09-12T03:19:26.742884+0000 front 2024-09-12T03:19:26.742923+0000 (oldest deadline 2024-09-12T03:19:50.843032+0000) 2024-09-12T03:19:52.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6806 osd.0 since back 2024-09-12T03:18:20.531976+0000 front 2024-09-12T03:18:20.532003+0000 (oldest deadline 2024-09-12T03:18:45.732450+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6804 osd.1 since back 2024-09-12T03:18:28.633508+0000 front 2024-09-12T03:18:28.633560+0000 (oldest deadline 2024-09-12T03:18:52.733483+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-12T03:18:33.834161+0000 front 2024-09-12T03:18:33.834190+0000 (oldest deadline 2024-09-12T03:18:59.134297+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6812 osd.4 since back 2024-09-12T03:18:51.837365+0000 front 2024-09-12T03:18:51.837335+0000 (oldest deadline 2024-09-12T03:19:16.537465+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-12T03:19:01.238327+0000 front 2024-09-12T03:19:01.238286+0000 (oldest deadline 2024-09-12T03:19:24.138417+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.57:6822 osd.6 since back 2024-09-12T03:19:11.140364+0000 front 2024-09-12T03:19:11.140319+0000 (oldest deadline 2024-09-12T03:19:32.840090+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.181:6820 osd.7 since back 2024-09-12T03:19:17.342162+0000 front 2024-09-12T03:19:17.342127+0000 (oldest deadline 2024-09-12T03:19:43.241994+0000) 2024-09-12T03:19:52.644 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:52 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11[60513]: 2024-09-12T03:19:52.334+0000 7f946426d640 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-09-12T03:19:26.742884+0000 front 2024-09-12T03:19:26.742923+0000 (oldest deadline 2024-09-12T03:19:50.843032+0000) 2024-09-12T03:19:54.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:53 smithi204 podman[72507]: 2024-09-12 03:19:53.736737174 +0000 UTC m=+5.612397464 container died df43a657e32d67c79b0d086f965c8def90078759d591db24c9184211d1952b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T03:19:54.358 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72507]: 2024-09-12 03:19:54.091490824 +0000 UTC m=+5.967151116 container cleanup df43a657e32d67c79b0d086f965c8def90078759d591db24c9184211d1952b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:19:54.359 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 bash[72507]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:19:54.643 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72522]: 2024-09-12 03:19:54.359323538 +0000 UTC m=+0.622317443 container remove df43a657e32d67c79b0d086f965c8def90078759d591db24c9184211d1952b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:19:54.962 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72610]: 2024-09-12 03:19:54.604082365 +0000 UTC m=+0.016026947 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:19:54.962 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72610]: 2024-09-12 03:19:54.718411336 +0000 UTC m=+0.130355909 container create 9eb02bda765b3549f6202b439dd919bde4fc31ec507ee0fbcbbfc65024d04476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:19:54.962 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72610]: 2024-09-12 03:19:54.963026979 +0000 UTC m=+0.374971554 container init 9eb02bda765b3549f6202b439dd919bde4fc31ec507ee0fbcbbfc65024d04476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:19:55.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:54 smithi204 podman[72610]: 2024-09-12 03:19:54.966366044 +0000 UTC m=+0.378310609 container start 9eb02bda765b3549f6202b439dd919bde4fc31ec507ee0fbcbbfc65024d04476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T03:19:55.393 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:55 smithi204 podman[72610]: 2024-09-12 03:19:55.063016998 +0000 UTC m=+0.474961563 container attach 9eb02bda765b3549f6202b439dd919bde4fc31ec507ee0fbcbbfc65024d04476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T03:19:55.394 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 12 03:19:55 smithi204 podman[72610]: 2024-09-12 03:19:55.153763631 +0000 UTC m=+0.565708197 container died 9eb02bda765b3549f6202b439dd919bde4fc31ec507ee0fbcbbfc65024d04476 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:19:55.717 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:19:55.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:19:55.756 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:19:55.757 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:19:55.757 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:19:56.119 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:19:55 smithi057 systemd[1]: Stopping Ceph mds.a for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:19:56.620 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:19:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a[71252]: 2024-09-12T03:19:56.162+0000 7f5c91400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:19:56.620 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:19:56 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a[71252]: 2024-09-12T03:19:56.162+0000 7f5c91400640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:20:00.505 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:20:00 smithi057 podman[93845]: 2024-09-12 03:20:00.19493636 +0000 UTC m=+4.140039385 container died b7e872f53d0d293a035cf094a05515b7e6b459439334fe65c1b8158d2692e4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:20:00.809 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:20:00 smithi057 podman[93845]: 2024-09-12 03:20:00.625651722 +0000 UTC m=+4.570754743 container cleanup b7e872f53d0d293a035cf094a05515b7e6b459439334fe65c1b8158d2692e4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-12T03:20:00.810 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:20:00 smithi057 bash[93845]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:20:01.066 INFO:journalctl@ceph.mds.a.smithi057.stdout:Sep 12 03:20:00 smithi057 podman[93857]: 2024-09-12 03:20:00.810088843 +0000 UTC m=+0.610204232 container remove b7e872f53d0d293a035cf094a05515b7e6b459439334fe65c1b8158d2692e4f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T03:20:01.075 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:20:01.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:01.148 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:20:01.148 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:20:01.148 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:20:01.369 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:01 smithi057 systemd[1]: Stopping Ceph mds.d for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:01.869 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d[69962]: 2024-09-12T03:20:01.574+0000 7f3520800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:01.870 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:01 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d[69962]: 2024-09-12T03:20:01.574+0000 7f3520800640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:20:04.869 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:04 smithi057 podman[93965]: 2024-09-12 03:20:04.47482478 +0000 UTC m=+3.005687657 container died f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:20:05.256 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:05 smithi057 podman[93965]: 2024-09-12 03:20:05.02188784 +0000 UTC m=+3.552750722 container cleanup f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T03:20:05.256 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:05 smithi057 bash[93965]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:20:05.543 INFO:journalctl@ceph.mds.d.smithi057.stdout:Sep 12 03:20:05 smithi057 podman[93988]: 2024-09-12 03:20:05.256480905 +0000 UTC m=+0.764650134 container remove f92c6bc0fbc0f54974ada223984098e6811b64f22cdc3a92aa551d8c95fb89d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:05.566 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:20:05.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:05.640 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:20:05.640 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:20:05.640 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:20:05.869 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:05 smithi057 systemd[1]: Stopping Ceph mds.g for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:06.369 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g[70845]: 2024-09-12T03:20:06.054+0000 7fde67600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:06.369 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:06 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g[70845]: 2024-09-12T03:20:06.054+0000 7fde67600640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:20:10.327 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:10 smithi057 podman[94094]: 2024-09-12 03:20:10.06717156 +0000 UTC m=+4.140728021 container died aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:20:10.601 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:10 smithi057 podman[94094]: 2024-09-12 03:20:10.441514603 +0000 UTC m=+4.515071064 container cleanup aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-12T03:20:10.601 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:10 smithi057 bash[94094]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:20:10.868 INFO:journalctl@ceph.mds.g.smithi057.stdout:Sep 12 03:20:10 smithi057 podman[94106]: 2024-09-12 03:20:10.601182742 +0000 UTC m=+0.520759410 container remove aa03573d3f411ae1a95edd1abb9960bf7200c2ffe4fcf5ed51879b029d3bad7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:20:10.898 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:20:10.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:10.969 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:20:10.970 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:20:10.970 DEBUG:teuthology.orchestra.run.smithi057:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:20:11.369 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:11 smithi057 systemd[1]: Stopping Ceph mds.j for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:11.869 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j[69589]: 2024-09-12T03:20:11.531+0000 7f7dcae00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:11.870 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:11 smithi057 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j[69589]: 2024-09-12T03:20:11.531+0000 7f7dcae00640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:20:15.846 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:15 smithi057 podman[94215]: 2024-09-12 03:20:15.546720489 +0000 UTC m=+4.250511769 container died c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T03:20:16.119 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:15 smithi057 podman[94215]: 2024-09-12 03:20:15.953402245 +0000 UTC m=+4.657193520 container cleanup c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T03:20:16.120 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:15 smithi057 bash[94215]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:20:16.389 INFO:journalctl@ceph.mds.j.smithi057.stdout:Sep 12 03:20:16 smithi057 podman[94227]: 2024-09-12 03:20:16.137598633 +0000 UTC m=+0.577564870 container remove c819ca720db65a676d893e87e7a6ce684e6637eb7061fd9cfd05530f7f21bb6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:20:16.395 DEBUG:teuthology.orchestra.run.smithi057:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:20:16.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:16.464 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:20:16.464 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:20:16.464 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:20:16.771 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:16 smithi181 systemd[1]: Stopping Ceph mds.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:17.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: 2024-09-12T03:20:16.769+0000 7fb832639640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:17.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:16 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b[65415]: 2024-09-12T03:20:16.769+0000 7fb832639640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:20:21.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:20 smithi181 podman[72407]: 2024-09-12 03:20:20.775773601 +0000 UTC m=+4.122043865 container stop e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:20:21.203 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:20 smithi181 podman[72407]: 2024-09-12 03:20:20.806024169 +0000 UTC m=+4.152294478 container died e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:20:21.504 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 podman[72407]: 2024-09-12 03:20:21.320347181 +0000 UTC m=+4.666617446 container cleanup e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T03:20:21.504 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 bash[72407]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:20:21.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:20:21.771 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 podman[72433]: 2024-09-12 03:20:21.504885996 +0000 UTC m=+0.721724544 container remove e2f6e23d0db73308d5bd6072498c7bab712256b01442c59b394c8f194c97311e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-b, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:20:21.771 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-12T03:20:21.771 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 systemd[1]: Stopped Ceph mds.b for 3c25d9c4-70b2-11ef-bcea-c7b262605968. 2024-09-12T03:20:21.771 INFO:journalctl@ceph.mds.b.smithi181.stdout:Sep 12 03:20:21 smithi181 systemd[1]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.b.service: Consumed 3.398s CPU time. 2024-09-12T03:20:21.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:21.786 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:20:21.786 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:20:21.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:20:22.124 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:21 smithi181 systemd[1]: Stopping Ceph mds.e for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:22.453 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e[64803]: 2024-09-12T03:20:22.123+0000 7f26bce79640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:22.453 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:22 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e[64803]: 2024-09-12T03:20:22.123+0000 7f26bce79640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:20:23.374 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:23 smithi181 podman[72540]: 2024-09-12 03:20:23.106519742 +0000 UTC m=+1.102743239 container died bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T03:20:23.703 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:23 smithi181 podman[72540]: 2024-09-12 03:20:23.465484108 +0000 UTC m=+1.461707603 container cleanup bdef0c4ed6214f8435c919b704e1965a224847c5b9536c8252adf6abdb1fa166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:20:23.703 INFO:journalctl@ceph.mds.e.smithi181.stdout:Sep 12 03:20:23 smithi181 bash[72540]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:20:23.889 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:20:23.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:23.922 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:20:23.922 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:20:23.922 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:20:24.453 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:24 smithi181 systemd[1]: Stopping Ceph mds.h for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:24.453 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: 2024-09-12T03:20:24.244+0000 7f9b00c6e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:24.453 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:24 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h[65103]: 2024-09-12T03:20:24.244+0000 7f9b00c6e640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:20:28.521 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:28 smithi181 podman[72659]: 2024-09-12 03:20:28.249484922 +0000 UTC m=+4.108336361 container died d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:20:29.112 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:28 smithi181 podman[72659]: 2024-09-12 03:20:28.85272844 +0000 UTC m=+4.711579884 container cleanup d00e9971ca12b807c858d75be7b1dcb960bbbf8673b6a1fa1ea75014accc8c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:20:29.112 INFO:journalctl@ceph.mds.h.smithi181.stdout:Sep 12 03:20:28 smithi181 bash[72659]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:20:29.272 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:20:29.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:29.304 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:20:29.305 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:20:29.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:20:29.703 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:29 smithi181 systemd[1]: Stopping Ceph mds.k for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:29.703 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k[64491]: 2024-09-12T03:20:29.614+0000 7f56a68dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:29.703 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:29 smithi181 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k[64491]: 2024-09-12T03:20:29.614+0000 7f56a68dc640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:20:32.949 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:32 smithi181 podman[72782]: 2024-09-12 03:20:32.636088492 +0000 UTC m=+3.116538509 container died b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:20:33.203 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:33 smithi181 podman[72782]: 2024-09-12 03:20:33.039316091 +0000 UTC m=+3.519766100 container cleanup b8b89a131a6c7430c86443fbaeed607b6daf2e7fed8125fc154f14cdb3c52a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:20:33.203 INFO:journalctl@ceph.mds.k.smithi181.stdout:Sep 12 03:20:33 smithi181 bash[72782]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:20:33.381 DEBUG:teuthology.orchestra.run.smithi181:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:20:33.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:33.414 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:20:33.414 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:20:33.414 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:20:33.717 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:33 smithi204 systemd[1]: Stopping Ceph mds.c for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:34.143 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: 2024-09-12T03:20:33.716+0000 7f5c7cde3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:34.143 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:33 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c[65435]: 2024-09-12T03:20:33.716+0000 7f5c7cde3640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:20:37.144 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:36 smithi204 podman[72725]: 2024-09-12 03:20:36.848164094 +0000 UTC m=+3.232343825 container died 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:37.475 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:37 smithi204 podman[72725]: 2024-09-12 03:20:37.266427647 +0000 UTC m=+3.650607380 container cleanup 4a24be7566a3f36cf26799f6b9e7a9a900793ec621f9517f6d80f1db77158132 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:20:37.475 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 12 03:20:37 smithi204 bash[72725]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:20:37.690 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:20:37.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:37.726 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:20:37.726 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:20:37.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:20:38.079 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:37 smithi204 systemd[1]: Stopping Ceph mds.f for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:38.393 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: 2024-09-12T03:20:38.079+0000 7f6a3d183640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:38.393 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:38 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f[65134]: 2024-09-12T03:20:38.079+0000 7f6a3d183640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:20:41.097 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:40 smithi204 podman[72847]: 2024-09-12 03:20:40.846204446 +0000 UTC m=+2.902491794 container died 0ece4f5feaf126fa82b98269f47095994019184c4fd2dae763df2e2c6b904d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:20:41.393 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:41 smithi204 podman[72847]: 2024-09-12 03:20:41.204119805 +0000 UTC m=+3.260407152 container cleanup 0ece4f5feaf126fa82b98269f47095994019184c4fd2dae763df2e2c6b904d2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:20:41.393 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 12 03:20:41 smithi204 bash[72847]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:20:41.631 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:20:41.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:41.664 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:20:41.664 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:20:41.664 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:20:41.893 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:41 smithi204 systemd[1]: Stopping Ceph mds.i for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:42.393 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: 2024-09-12T03:20:41.999+0000 7f6c01099640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:42.393 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:42 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i[65738]: 2024-09-12T03:20:41.999+0000 7f6c01099640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:20:45.143 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:44 smithi204 podman[72966]: 2024-09-12 03:20:44.845084093 +0000 UTC m=+2.968223478 container died 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:20:45.584 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:45 smithi204 podman[72966]: 2024-09-12 03:20:45.300104786 +0000 UTC m=+3.423244168 container cleanup 2bb60bfc56839eff95b4f65b2d7c1ecb64fc2e77014a7b34c00c70b4301bb7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T03:20:45.584 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 12 03:20:45 smithi204 bash[72966]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:20:45.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:20:45.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:45.763 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:20:45.763 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:20:45.763 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:20:46.143 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:45 smithi204 systemd[1]: Stopping Ceph mds.l for 3c25d9c4-70b2-11ef-bcea-c7b262605968... 2024-09-12T03:20:46.143 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l[64829]: 2024-09-12T03:20:46.079+0000 7efe05127640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:20:46.143 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:46 smithi204 ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l[64829]: 2024-09-12T03:20:46.079+0000 7efe05127640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:20:50.180 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:49 smithi204 podman[73099]: 2024-09-12 03:20:49.90075987 +0000 UTC m=+3.926200722 container died a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:20:50.538 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:50 smithi204 podman[73099]: 2024-09-12 03:20:50.283629845 +0000 UTC m=+4.309070695 container cleanup a3101db400f951b9f5fd75ee529ba4ad3dc37eb18caaa15b4babb326d8b6855f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T03:20:50.538 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 12 03:20:50 smithi204 bash[73099]: ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:20:50.665 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c25d9c4-70b2-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:20:50.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:20:50.698 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:20:50.698 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:20:50.979 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:20:55.037 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:20:55.237 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:20:57.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:20:58.111 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:21:00.511 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:21:00.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:21:00.610 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:21:00.637 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:21:00.638 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi057/crash 2024-09-12T03:21:00.639 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:21:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:tar: /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:21:00.696 INFO:teuthology.orchestra.run.smithi057.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:21:00.699 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi181/crash 2024-09-12T03:21:00.700 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:21:00.743 INFO:teuthology.orchestra.run.smithi181.stderr:tar: /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:21:00.744 INFO:teuthology.orchestra.run.smithi181.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:21:00.745 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi204/crash 2024-09-12T03:21:00.746 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:21:00.786 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:21:00.786 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:21:00.788 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:21:00.788 DEBUG:teuthology.orchestra.run.smithi057:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-12T03:21:00.921 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:21:00.921 DEBUG:teuthology.orchestra.run.smithi057:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:21:00.925 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:21:00.928 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:21:00.952 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:21:00.952 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-09-12T03:21:00.953 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:21:00.953 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:21:00.956 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:21:00.956 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:21:00.956 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:21:00.956 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:21:00.956 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip 81.7% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:21:00.957 INFO:teuthology.orchestra.run.smithi204.stderr: -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:21:00.957 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:21:00.957 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:21:00.957 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:21:00.958 INFO:teuthology.orchestra.run.smithi181.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:21:00.958 INFO:teuthology.orchestra.run.smithi181.stderr:: No such file or directory 2024-09-12T03:21:00.958 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:21:00.958 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:21:00.959 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:21:00.962 INFO:teuthology.orchestra.run.smithi204.stderr: 86.9% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:21:00.962 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:21:00.962 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.b.log: /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.4.log: /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:21:00.964 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log: 94.5% 81.7% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:21:00.965 INFO:teuthology.orchestra.run.smithi181.stderr: -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:21:00.965 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:21:00.965 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:21:00.966 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:21:00.966 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:21:00.966 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.z.log: 91.0% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:21:00.967 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:21:00.967 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:21:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.h.log: 91.0% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:21:00.971 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:21:00.973 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:21:00.975 INFO:teuthology.orchestra.run.smithi204.stderr: 92.1% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:21:00.976 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:21:00.976 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:21:00.986 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:21:00.990 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:21:00.993 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:21:01.002 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:21:01.005 INFO:teuthology.orchestra.run.smithi057.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-12T03:21:01.005 INFO:teuthology.orchestra.run.smithi057.stderr:: No such file or directory 2024-09-12T03:21:01.005 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:21:01.006 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:21:01.007 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:21:01.008 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.9.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-12T03:21:01.008 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:21:01.009 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:21:01.009 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:21:01.010 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:21:01.011 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:21:01.012 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:21:01.012 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:21:01.013 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:21:01.014 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:21:01.015 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:21:01.016 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:21:01.016 INFO:teuthology.orchestra.run.smithi057.stderr: 86.9% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:21:01.017 INFO:teuthology.orchestra.run.smithi057.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:21:01.017 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:21:01.019 INFO:teuthology.orchestra.run.smithi057.stderr:/var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.2% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:21:01.021 INFO:teuthology.orchestra.run.smithi057.stderr: 94.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:21:01.045 INFO:teuthology.orchestra.run.smithi204.stderr: 94.6% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:21:01.070 INFO:teuthology.orchestra.run.smithi181.stderr: 93.2% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:21:01.106 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:21:01.226 INFO:teuthology.orchestra.run.smithi057.stderr: 90.2% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:21:01.293 INFO:teuthology.orchestra.run.smithi057.stderr: 91.0% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:21:01.302 INFO:teuthology.orchestra.run.smithi181.stderr: 92.7% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:21:01.319 INFO:teuthology.orchestra.run.smithi057.stderr: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:21:01.321 INFO:teuthology.orchestra.run.smithi057.stderr: 84.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:21:01.424 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:21:01.509 INFO:teuthology.orchestra.run.smithi181.stderr: 91.8% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:21:01.513 INFO:teuthology.orchestra.run.smithi204.stderr: 91.6% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:21:01.554 INFO:teuthology.orchestra.run.smithi057.stderr: 92.8% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:21:02.242 INFO:teuthology.orchestra.run.smithi057.stderr: 90.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:21:05.130 INFO:teuthology.orchestra.run.smithi057.stderr: 90.5% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:21:24.437 INFO:teuthology.orchestra.run.smithi181.stderr: 92.6% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:21:33.365 INFO:teuthology.orchestra.run.smithi204.stderr: 92.5% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:21:36.234 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:21:42.883 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:21:43.725 INFO:teuthology.orchestra.run.smithi181.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:21:50.373 INFO:teuthology.orchestra.run.smithi181.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:21:53.261 INFO:teuthology.orchestra.run.smithi181.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:21:53.263 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-09-12T03:21:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m52.320s 2024-09-12T03:21:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:user 2m43.327s 2024-09-12T03:21:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m5.368s 2024-09-12T03:22:05.272 INFO:teuthology.orchestra.run.smithi204.stderr: 92.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:22:05.274 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-12T03:22:05.274 INFO:teuthology.orchestra.run.smithi204.stderr:real 1m4.332s 2024-09-12T03:22:05.274 INFO:teuthology.orchestra.run.smithi204.stderr:user 2m49.395s 2024-09-12T03:22:05.274 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m5.489s 2024-09-12T03:23:51.434 INFO:teuthology.orchestra.run.smithi057.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:23:56.063 INFO:teuthology.orchestra.run.smithi057.stderr: 92.4% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:24:01.764 INFO:teuthology.orchestra.run.smithi057.stderr: 92.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:24:19.419 INFO:teuthology.orchestra.run.smithi057.stderr: 92.3% -- replaced with /var/log/ceph/3c25d9c4-70b2-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:24:19.422 INFO:teuthology.orchestra.run.smithi057.stderr: 2024-09-12T03:24:19.422 INFO:teuthology.orchestra.run.smithi057.stderr:real 3m18.468s 2024-09-12T03:24:19.422 INFO:teuthology.orchestra.run.smithi057.stderr:user 3m32.902s 2024-09-12T03:24:19.422 INFO:teuthology.orchestra.run.smithi057.stderr:sys 0m21.696s 2024-09-12T03:24:19.423 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:24:19.424 DEBUG:teuthology.misc:Transferring archived files from smithi057:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi057/log 2024-09-12T03:24:19.425 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:24:52.065 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi181/log 2024-09-12T03:24:52.066 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:25:19.033 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi204/log 2024-09-12T03:25:19.034 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:25:46.518 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:25:46.518 DEBUG:teuthology.orchestra.run.smithi057:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force 2024-09-12T03:25:46.804 INFO:teuthology.orchestra.run.smithi057.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:25:47.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force 2024-09-12T03:25:48.178 INFO:teuthology.orchestra.run.smithi181.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:25:48.824 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c25d9c4-70b2-11ef-bcea-c7b262605968 --force 2024-09-12T03:25:49.031 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 3c25d9c4-70b2-11ef-bcea-c7b262605968 2024-09-12T03:25:49.592 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:25:49.592 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:25:49.634 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:25:49.650 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:25:49.667 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:25:49.667 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:25:49.677 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:25:49.678 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:25:49.682 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:25:49.693 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:25:49.841 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:25:49.859 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:25:49.859 DEBUG:teuthology.orchestra.run.smithi057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:25:49.864 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:25:49.867 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:25:49.879 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:25:49.882 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:25:49.883 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:25:49.883 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2024-09-12T03:25:49.884 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -2938us[-2960us] +/- 46ms 2024-09-12T03:25:49.884 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 +1830us[+1830us] +/- 61ms 2024-09-12T03:25:49.884 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +2608us[+2608us] +/- 72ms 2024-09-12T03:25:49.884 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:25:49.886 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:25:49.886 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-12T03:25:49.887 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -2906us[-2918us] +/- 46ms 2024-09-12T03:25:49.887 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 +1821us[+1821us] +/- 61ms 2024-09-12T03:25:49.887 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 86 +2634us[+2614us] +/- 71ms 2024-09-12T03:25:49.887 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:25:49.895 INFO:teuthology.orchestra.run.smithi057.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:25:49.902 INFO:teuthology.orchestra.run.smithi057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:25:49.902 INFO:teuthology.orchestra.run.smithi057.stdout:=============================================================================== 2024-09-12T03:25:49.902 INFO:teuthology.orchestra.run.smithi057.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -2922us[-2937us] +/- 46ms 2024-09-12T03:25:49.903 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +1842us[+1842us] +/- 61ms 2024-09-12T03:25:49.903 INFO:teuthology.orchestra.run.smithi057.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +2618us[+2618us] +/- 72ms 2024-09-12T03:25:49.903 INFO:teuthology.orchestra.run.smithi057.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:25:49.904 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:25:49.921 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:25:49.922 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:25:49.938 DEBUG:teuthology.orchestra.run.smithi057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:25:50.101 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:25:50.168 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:25:50.241 DEBUG:teuthology.orchestra.run.smithi057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:25:50.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:50.303 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:25:50.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:50.331 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:25:50.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:50.359 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:25:50.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:25:50.380 INFO:teuthology.task.internal:Duration was 2315.688749 seconds 2024-09-12T03:25:50.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:25:50.390 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:25:50.390 DEBUG:teuthology.orchestra.run.smithi057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:25:50.395 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:25:50.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:25:50.433 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:25:50.437 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:25:50.493 INFO:teuthology.orchestra.run.smithi057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:25:50.744 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:25:50.744 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi057.front.sepia.ceph.com 2024-09-12T03:25:50.744 DEBUG:teuthology.orchestra.run.smithi057:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:25:50.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-09-12T03:25:50.794 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:25:50.817 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-09-12T03:25:50.817 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:25:50.855 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:25:50.855 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:25:50.859 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:25:50.862 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:25:50.921 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:25:50.921 DEBUG:teuthology.orchestra.run.smithi057:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:25:50.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:25:50.929 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:25:51.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:25:51.648 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:25:51.648 DEBUG:teuthology.orchestra.run.smithi057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:25:51.716 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:25:51.743 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:25:51.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:25:51.786 DEBUG:teuthology.orchestra.run.smithi057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.790 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.792 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.815 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-09-12T03:25:51.836 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-09-12T03:25:51.871 INFO:teuthology.orchestra.run.smithi057.stdout:kernel.core_pattern = core 2024-09-12T03:25:51.901 DEBUG:teuthology.orchestra.run.smithi057:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:51.936 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:51.951 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:25:51.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:25:51.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:25:51.977 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:25:51.978 DEBUG:teuthology.misc:Transferring archived files from smithi057:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi057 2024-09-12T03:25:51.979 DEBUG:teuthology.orchestra.run.smithi057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:25:52.059 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi181 2024-09-12T03:25:52.059 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:25:52.096 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901648/remote/smithi204 2024-09-12T03:25:52.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:25:52.134 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:25:52.134 DEBUG:teuthology.orchestra.run.smithi057:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:25:52.138 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:25:52.140 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:25:52.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:25:52.196 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:25:52.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:25:52.209 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:25:52.209 DEBUG:teuthology.orchestra.run.smithi057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:25:52.214 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:25:52.216 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:25:52.228 INFO:teuthology.orchestra.run.smithi181.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:25 /home/ubuntu/cephtest 2024-09-12T03:25:52.240 INFO:teuthology.orchestra.run.smithi204.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:25 /home/ubuntu/cephtest 2024-09-12T03:25:52.245 INFO:teuthology.orchestra.run.smithi057.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:25 /home/ubuntu/cephtest 2024-09-12T03:25:52.247 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:25:52.259 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:25:52.310 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} duration: 2315.6887493133545 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:25:52.311 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:25:52.403 INFO:teuthology.run:pass